CONTESTANT BEHAVIOUR AND PERFORMATIVEREPUTATION: EVIDENCE FROM CROWDSOURCING OPEN INNOVATION TOURNAMENTS
DOI:
https://doi.org/10.69980/xk9td776Keywords:
Crowdsourcing contests, platform, innovation tournaments, open innovationAbstract
Crowdsourcing open innovation tournaments are online platforms that organizers use to expose tasks to a large number of individuals. This study examines the impact of contestants' participation behaviour dimensions on the performative reputation component in a crowdsourced innovation platform (Kaggle.com), incorporating a performance-based reputation element that allows users to track competitors unilaterally. Using Expectancy and Self-Determination Theory and empirically studying cross-sectional data from 523,797 contestants in 2015, we found that active frequency has a consistent positive impact on contestants' ratings and badges, while late submission behaviour negatively affects rating and badge performance. While more votes increase the frequency of contestants' activity and reduce late submissions, comments, in combination, have the opposite effect on contestants' rating points but a more significant impact on badges relative to followers' votes. These findings have significant implications for software developers and innovation platform managers, both in theory and in practice.
References
1.Anderson, A., Huttenlocher, D., Kleinberg, J., and Leskovec, J. (2014) ‘Engaging with massive online courses’, Proceedings of the 23rd international conference on World wide web, pp. 687– 698. doi:10.1145/2566486.2568042.
2.Araujo, R. (2013) ‘99designs: An Analysis of Creative Competition in crowdsourced design’, Proceedings of the AAAI Conference on Human Computation and Crowdsourcing, Vol. 1, pp. 17– 24. doi:10.1609/hcomp.v1i1.13081.
3.Archak, N. (2010) ‘Money, glory and cheap talk’, Proceedings of the 19th international conference on World wide web, pp. 21–30. doi:10.1145/1772690.1772694.
4.Backes‐Gellner, U. and Pull, K. (2013) ‘Tournament compensation systems, employee heterogeneity, and firm performance’, Human Resource Management, Vol. 52, No. 3, pp. 375– 398. doi:10.1002/hrm.21535.
5.Baruch, A., May, A. and Yu, D. (2016) ‘The motivations, enablers and barriers for voluntary participation in an online crowdsourcing platform’, Computers in Human Behavior, Vol. 64, pp. 923–931. doi:10.1016/j.chb.2016.07.039.
6.Beal DJ, Cohen RR, Burke MJ, McLendon CL (2003) ‘Cohesion and performance in groups: A meta-analytic clarification of construct relations.’, Journal of Applied Psychology, Vol. 88 No. 6, pp. 989–1004. doi:10.1037/0021-9010.88.6.989.
7.Boudreau, K.J., Lacetera, N. and Lakhani, K.R. (2011) ‘Incentives and problem uncertainty in innovation contests: An empirical analysis’, Management Science, Vol. 57, No. 5, pp. 843–863. doi:10.1287/mnsc.1110.1322.
8.Boudreau, K.J., Lakhani, K.R. and Menietti, M. (2016) ‘Performance responses to competition across skill levels in rank‐order tournaments: Field evidence and implications for tournament design’, The RAND Journal of Economics, Vol. 47 No. 1, pp. 140–165. doi:10.1111/1756- 2171.12121.
9.Bouvier, D., Lovellette, E. and Matta, J. (2021) ‘Overnight feedback reduces late submissions on programming projects in CS1’, Proceedings of the 23rd Australasian Computing Education Conference, pp. 176–180. doi:10.1145/3441636.3442319.
10.Burtch, G., He, Q., Hong, Y., and Lee, D. (2022) ‘How do peer awards motivate creative content? experimental evidence from reddit’, Management Science, Vol. 68, No. 5, pp. 3488–3506. doi:10.1287/mnsc.2021.4040.
11.Cai, Y. and Zhu, D. (2016) ‘Reputation in an open source software community: Antecedents and impacts’, Decision Support Systems, Vol. 91, pp. 103–112. doi:10.1016/j.dss.2016.08.004.
12.Carpenter D. (2010) ‘Chapter One. reputation and regulatory power’ (2014) Reputation and Power, pp. 33–70. doi:10.1515/9781400835119.33.
13.Cassiman, B. and Veugelers, R. (2006) ‘In search of complementarity in innovation strategy: Internal R&D and External Knowledge Acquisition’, Management Science, Vol. 52 No. 1, pp. 68– 82. doi:10.1287/mnsc.1050.0470.
14.Chambers, C.R. (2024) ‘Nonmonetary reward systems, counterproductive behavior, and responses to sanctions in open collaboration environments’, Organisation Science, Vol. 35, No. 3, pp. 928– 947. doi:10.1287/orsc.2020.14548.
15.Chua, R.Y., Roth, Y. and Lemoine, J.-F. (2014) ‘The impact of culture on creativity’,Administrative Science Quarterly, Vol. 60, No. 2, pp. 189–227. doi:10.1177/0001839214563595.
16.Deodhar, S.J. and Gupta, S. (2023) ‘The impact of social reputation features in innovation tournaments: Evidence from a natural experiment’, Information Systems Research, Vol. 34, No. 1, pp. 178–193. doi:10.1287/isre.2022.1118.
17.de Beer J, McCarthy IP, Soliman A, Treen E (2017) ‘Click here to agree: Managing intellectual property when crowdsourcing solutions’, Business Horizons, Vol. 60 No. 2, pp. 207–217. doi:10.1016/j.bushor.2016.11.002.
18.Deichmann, D., Gillier, T. and Tonellato, M. (2021) ‘Getting on board with new ideas: An analysis of idea commitments on a crowdsourcing platform’, Research Policy, Vol. 50 No. 9, p. 104320. doi:10.1016/j.respol.2021.104320.
19.Dissanayake, I., Zhang, J., Yasar, M., and Nerur, S. P. (2018) ‘Strategic effort allocation in online innovation tournaments’, Information and Management, Vol. 55, No. 3, pp. 396–406. doi:10.1016/j.im.2017.09.006.
20.Faraj, S. and Johnson, S.L. (2011) ‘Network exchange patterns in online communities’,Organisation Science, Vol. 22, No. 6, pp. 1464–1480. doi:10.1287/orsc.1100.0600.
21.Feuerverger, A., He, Y. and Khatri, S. (2012) ‘Statistical significance of the netflix challenge’,Statistical Science, Vol. 27, No. 2. doi:10.1214/11-sts368.
22.Gamber, M., Kruft, T. and Kock, A. (2021) ‘Which effort pays off? analyzing ideators’ behavioral patterns on corporate ideation platforms’, Journal of Product Innovation Management, Vol. 39, No. 3, pp. 419–444. doi:10.1111/jpim.12593.
23.Garcia Martinez, M. (2017) ‘Inspiring crowdsourcing communities to create novel solutions: Competition design and the mediating role of trust’, Technological Forecasting and Social Change, Vol. 117, pp. 296–304. doi:10.1016/j.techfore.2016.11.015.
24.Goes, P.B., Lin, M. and Au Yeung, C. (2014) ‘“popularity effect” in user-generated content: Evidence from online product reviews’, Information Systems Research, Vol. 25 No. 2, pp. 222– 238. doi:10.1287/isre.2013.0512.
25.Gupta, K., Su, Y., Kunkel, T., and Funk, D. C. (2024) ‘Paying while playing: Examining the influence of interaction with gamified elements in fantasy sports on in-app spending’, European Sport Management Quarterly, Vol. 25, No. 2, pp. 175–196. doi:10.1080/16184742.2024.2301970.
26.Hammer B (2017) Feature launch: Follow your favorite Kaggle Users | Kaggle. Retrieved January 25, 2019, https://www. kaggle.com/general/33512
27.Hertel, G., Niedner, S. and Herrmann, S. (2003) ‘Motivation of software developers in open source projects: An internet-based survey of contributors to the linux kernel’, Research Policy, Vol. 32, No. 7, pp. 1159–1177. doi:10.1016/s0048-7333(03)00047-7.
28.Hopkins, E. (2018) ‘Inequality and risk-taking behaviour’, Games and Economic Behavior, Vol. 107, pp. 316–328. doi:10.1016/j.geb.2017.11.007.
29.Huang, B., Hew, K.F. and Lo, C.K. (2018) ‘Investigating the effects of gamification-enhanced flipped learning on undergraduate students’ behavioral and cognitive engagement’, Interactive Learning Environments, Vol. 27, No. 8, pp. 1106–1126. doi:10.1080/10494820.2018.1495653.
30.Hu, F., Huizingh, E. and Bijmolt, T. (2022) ‘Innovation contests: Attracting new solvers and New High‐Quality Solutions’, R&D Management, Vol. 53, No. 1, pp. 149–167. doi:10.1111/radm.12553.
31.Hutter, K., J. Füller, J. Hautz, V. Bilgram, and K. Matzler. (2015) ‘Machiavellianism or morality: Which behavior pays off in online innovation contests?’, Journal of Management Information Systems, Vol. 32, No. 3, pp. 197–228. doi:10.1080/07421222.2015.1099181.
32.Jabr, W., Mookerjee, R., Tan, Y., and Mookerjee, V. S. (2014) ‘Leveraging philanthropic behavior for customer support: The case of user support forums1’, MIS Quarterly, Vol. 38, No. 1, pp. 187– 208. doi:10.25300/misq/2014/38.1.09.
33.Jain, S. and Deodhar, S.J. (2021) ‘Social mechanisms in crowdsourcing contests: A literature review’, Behaviour and Information Technology, Vol. 41, No. 5, pp. 1080–1114. doi:10.1080/0144929x.2021.1880638.
34.Jeppesen, L.B. and Lakhani, K.R. (2010) ‘Marginality and problem-solving effectiveness in broadcast search’, Organisation Science, Vol. 21, No. 5, pp. 1016–1033. doi:10.1287/orsc.1090.0491.
35.Jo, H. and Bang, Y. (2023) ‘Retracted article: Factors influencing continuance intention of participants in crowdsourcing’, Humanities and Social Sciences Communications, Vol. 10, No. 1. doi:10.1057/s41599-023-02335-0.
36.Kapse, M., Jain, P. S., Pathak, A., Poulose, J., and Sharma, V. (2025) ‘Intrinsic motivational factors and creative and innovative work behaviour of employees: The mediating effect of creative personality’, International Journal of Indian Culture and Business Management, Vol. 34 No. 4, pp. 481–503. doi:10.1504/ijicbm.2025.145669.
37.Ke, C., Kubitz, G., Liu, Y., and Page, L. (2021). How tournament incentives shape risk-taking decisions.WorkingPaper. https://www.gskubitz.com/uploads/6/2/6/3/62634961/risk_taking_in_rank_order_contest_paper_ ii 1_.pdf.
38.Kokkodis, M. and Ipeirotis, P.G. (2016) ‘Reputation transferability in online labor markets’,Management Science, Vol. 62, No. 6, pp. 1687–1706. doi:10.1287/mnsc.2015.2217.
39.Korpeoglu, C.G., Körpeoğlu, E. and Tunç, S. (2021) ‘Optimal duration of innovation contests’, Manufacturing and Service Operations Management, Vol. 23, No. 3, pp. 657–675. doi:10.1287/msom.2020.0935.
40.Korpusik, M., Freitas, J. and Dionisio, J.D. (2022) ‘Impact of late policies on submission behavior and grades in Computer Programming’, 2022 ASEE Annual Conference and Exposition Proceedings [Preprint]. doi:10.18260/1-2--41566.Lakhani, K.R. and Wolf, R.G. (2005) ‘Why hackers do what they do: Understanding motivation and effort in free/open source software projects’, Perspectives on Free and Open Source Software, pp. 3–22. doi:10.7551/mitpress/5326.003.0005.
41.Liu, D., Santhanam, R. and Webster, J. (2017) ‘Toward meaningful engagement: A framework for design and research of gamified information systems1’, MIS Quarterly, Vol. 41, No. 4, pp. 1011– 1034. doi:10.25300/misq/2017/41.4.01.
42.Liu, L. and Munro, M. (2012) ‘Systematic analysis of centralized online reputation systems’,Decision Support Systems, Vol. 52, No. 2, pp. 438–449. doi:10.1016/j.dss.2011.10.003.
43.Liu, Z., Hatton, M. R., Kull, T., Dooley, K., and Oke, A. (2020) ‘Is a large award truly attractive to solvers? the impact of award size on crowd size in innovation contests’, Journal of Operations Management, Vol. 67, No. 4, pp. 420–449. doi:10.1002/joom.1132.
44.Loh, J. and Kretschmer, T. (2022) ‘Online communities on competing platforms: Evidence from game wikis’, Strategic Management Journal, Vol. 44, No. 2, pp. 441–476. doi:10.1002/smj.3442.
45.Luca, M. and Zervas, G. (2016) ‘Fake it till you make it: Reputation, competition, and Yelp Review Fraud’, Management Science, Vol. 62, No. 12, pp. 3412–3427. doi:10.1287/mnsc.2015.2304.
46.Martina, A. and Nagarajan, P.S. (2022) ‘Influence of organisational citizenship behaviour on the individual work performance of IT and ites employees’, International Journal of Indian Culture and Business Management, Vol. 27, No. 4, p. 421. doi:10.1504/ijicbm.2022.127725.
47.McLure Wasko, M. and Faraj, S. (2000) ‘“it is what one does”: Why people participate and help others in electronic communities of Practice’, The Journal of Strategic Information Systems, Vol. 9, No. 2–3, pp. 155–173. doi:10.1016/s0963-8687(00)00045-7.
48.Meeus, M.T.H. and Oerlemans, L.A.G. (2000) ‘Firm behaviour and innovative performance’,Research Policy, Vol. 29, No. 1, pp. 41–58. doi:10.1016/s0048-7333(99)00032-3.
49.Mihm, J. and Schlapp, J. (2019) ‘Sourcing innovation: On feedback in contests’, Management Science, 65(2), pp. 559–576. doi:10.1287/mnsc.2017.2955.
50.Mohanty, S., Dash, M., Naveen, L., Acharya, A., Sharma, D., and Muduli, K. (2025) ‘Emerging patterns in socially responsible investment: An empirical study of Indian it workers’ behavioural intention’, International Journal of Indian Culture and Business Management, Vol. 36 No. 1, pp. 30–55. doi:10.1504/ijicbm.2025.148412.
51.Moldovanu, B. and Sela, A. (2001) ‘The optimal allocation of prizes in contests’, American Economic Review, Vol. 91, No. 3, pp. 542–558. doi:10.1257/aer.91.3.542.
52.Moqri M, Mei X, Qiu L, Bandyopadhyay S (2018) ‘Effect of “following” on contributions to open source communities’, Journal of Management Information Systems, Vol. 35 No. 4, pp. 1188–1217. doi:10.1080/07421222.2018.1523605.
53.Mustafa, S., Zhang, W. and Naveed, M.M. (2023) ‘How to mend the dormant user in Q&A communities? A social cognitive theory-based study of consistent geeks of stackoverflow’, Behaviour and Information Technology, Vol. 43 No. 10, pp. 2024–2043. doi:10.1080/0144929x.2023.2237604.Namasudra, S. and Sharma, P. (2023) ‘Achieving a decentralized and secure cab sharing system using blockchain technology’, IEEE Transactions on Intelligent Transportation Systems, Vol. 24, No. 12, pp. 15568–15577. doi:10.1109/tits.2022.3186361.
54.Nanda, P.K. and Nagasubramaniyan, G. (2025) ‘A comprehensive employee performance enhancement model for Indian organised sectors’, International Journal of Indian Culture and Business Management, Vol. 36 No. 2, pp. 161–186. doi:10.1504/ijicbm.2025.148924.
55.O’Mahony, S. and Karp, R. (2020) ‘From proprietary to collective governance: How do platform participation strategies evolve?’, Strategic Management Journal, Vol. 43, No. 3, pp. 530–562. doi:10.1002/smj.3150.
56.Pandey, N., Gupta, A. and Mahajan, H. (2024) ‘A comprehensive study on factors of work life balance and big five personality traits and their impact on perceived performance of employees’,
57.International Journal of Indian Culture and Business Management, Vol. 32 No. 4, pp. 547–565. doi:10.1504/ijicbm.2024.140337.
58.Park, G., Oh, H., Lim, B. C., and Khoo, B. L. (2022) ‘Can smart technology make group members more creative? The effect of interactive feedback using sociometric badges on members’ creativity, Behaviour and Information Technology, Vol. 42, No. 14, pp. 2452–2466. doi:10.1080/0144929x.2022.2126949.
59.Pollok, P., Lüttgens, D. and Piller, F.T. (2019) ‘Attracting solutions in crowdsourcing contests: The role of knowledge distance, identity disclosure, and seeker status’, Research Policy, Vol. 48, No. 1, pp. 98–114. doi:10.1016/j.respol.2018.07.022.
60.Restivo, M. and van de Rijt, A. (2014) ‘No praise without effort: Experimental evidence on how rewards affect Wikipedia’s contributor community’, Information, Communication and Society, Vol. 17, No. 4, pp. 451–462. doi:10.1080/1369118x.2014.888459.
61.Riedl, C. and Woolley, A.W. (2017) ‘Teams vs. crowds: A field test of the relative contribution of incentives, member ability, and emergent collaboration to crowd-based problem solving performance’, Academy of Management Discoveries, Vol. 3 No. 4, pp. 382–403. doi:10.5465/amd.2015.0097.
62.Rigtering, J.P.C. (Coen), Weitzel, G.U. (Utz) and Muehlfeld, K. (Katrin) (2019) ‘Increasing quantity without compromising quality: How managerial framing affects intrapreneurship’, Journal of Business Venturing, Vol. 34, No. 2, pp. 224–241. doi:10.1016/j.jbusvent.2018.11.002.
63.Ryan, C., Wapnick, J., Lacaille, N., and Darrow, A. A. (2006) ‘The effects of various physical characteristics of high-level performers onadjudicators’ performance ratings’, Psychology of Music, Vol. 34 No. 4, pp. 559–572. doi:10.1177/0305735606068106.
64.Schenk, E., Guittard, C. and Pénin, J. (2019) ‘Open or proprietary? choosing the right crowdsourcing platform for Innovation’, Technological Forecasting and Social Change, Vol. 144, pp. 303–310. doi:10.1016/j.techfore.2017.11.021.
65.Schilling, M.A. (2009) ‘Protecting or diffusing a technology platform: Tradeoffs in appropriability, network externalities, and architectural control’, Platforms, Markets and Innovation [Preprint]. doi:10.4337/9781849803311.00015.
66.Schneider, J., Rosman, T., Kelava, A., and Merk, S. (2022) ‘Do open-science badges increase trust in scientists among undergraduates, scientists, and the public?’, Psychological Science, Vol. 33, No. 9, pp. 1588–1604. doi:10.1177/09567976221097499.
67.Segev, E. (2020) ‘Crowdsourcing contests’, European Journal of Operational Research, Vol. 281, No. 2, pp. 241–255. doi:10.1016/j.ejor.2019.02.057.
68.Shah, S.K. (2006) ‘Motivation, governance, and the viability of hybrid forms in open source software development’, Management Science, Vol. 52, No. 7, pp. 1000–1014. doi:10.1287/mnsc.1060.0553.
69.Singh, N., Pawar, A., Shah, R. G., Thakkar, T., Jani, A., and Vora, H. (2025) ‘Measuring the impacts of organisational culture determinants on organisational citizenship behaviour in it sector’, International Journal of Indian Culture and Business Management, Vol. 35 No. 3, pp. 328–343. doi:10.1504/ijicbm.2025.147155.
70.Sisak, D. (2009) ‘Multiple‐prize contests – the optimal allocation of prizes’, Journal of Economic Surveys, Vol. 23, No. 1, pp. 82–114. doi:10.1111/j.1467-6419.2008.00557.x.
71.Stewart, D. (2005) ‘Social status in an open-source community’, American Sociological Review, Vol. 70, No. 5, pp. 823–842. doi:10.1177/000312240507000505.
72.Vroom, V. H. (1964). Work and motivation. Xiao, Y. and Hew, K.F. (2023) ‘Intangible rewards versus tangible rewards in gamified online learning: Which promotes student intrinsic motivation, behavioural engagement, cognitive engagement and learning performance?’, British Journal of Educational Technology, Vol. 55, No. 1, pp. 297–317. doi:10.1111/bjet.13361.
73.Xu, L., Nian, T. and Cabral, L. (2020) ‘What makes geeks tick? A study of Stack overflow careers’,Management Science, Vol. 66, No. 2, pp. 587–604. doi:10.1287/mnsc.2018.3264.
74.Yang, Y., P. Y. Chen, and R. Banker. (2010) “Impact of Past Performance and Strategic Bidding on Winner Determination of Open Innovation Contest.” In Workshop on Information Systems and Economics, pp.11–12.
75.Ye, J. (Hua) and Jensen, M. (2022) ‘Effects of introducing an online community in a crowdsourcing contest platform’, Information Systems Journal, Vol. 32 No. 6, pp. 1203–1230. doi:10.1111/isj.12397.
76.Ye, S., Gao, G. (Gordon) and Viswanathan, S. (2014) ‘Strategic behavior in online reputation systems: Evidence from revoking on ebay1’, MIS Quarterly, Vol. 38, No. 4, pp. 1033–1056. doi:10.25300/misq/2014/38.4.05.
77.Zaggl, M.A. (2017) ‘Manipulation of explicit reputation in innovation and knowledge exchange communities: The example of referencing in science’, Research Policy, Vol. 46, No. 5, pp. 970– 983. doi:10.1016/j.respol.2017.02.009.
78.Zhang, S. and Singh, P.V. (2016) ‘A structural analysis of the role of superstars in crowdsourcing contests’, SSRN Electronic Journal [Preprint]. doi:10.2139/ssrn.2764553.
