2017 was a year of growth for MetaBrainz, complete with some growing pains.
The year started off with us hiring Elizabeth Bigger, aka Quesito, as our supporter catalyst. Her job involved being our primary contact for supporters, on-boarding new supporters, sending invoices and following up on unpaid invoices. In February we also finished off another successful participation in Google Code-in, that resulted in Anshuman Agarwal and Daniel Hsing becoming Grand Prize winners! Congratulations to both of you! In the same month we also got accepted to another round of GSoC!!
In March we hired Yvan Rivierre from France as another contractor to work on MusicBrainz with Bitmap. Yvan had been submitting various new features to MusicBrainz previously, so he was not new to our team, making it easy to bring him on board. April was pretty quiet, with just a Picard release and a MusicBrainz server release; however, most of our time was spent preparing for our schema change release, which happened on May 15 as scheduled.
In June we took an uncharacteristic step and supported a Kickstarter that was near and dear to us. Our friends from MuseScore created a Kickstarter project to liberate several pieces of sheet music and to place these transcriptions into the public domain. We managed to sponsor the transcription of Beethoven’s 5th!
With the help of Param Singh (aka iliekcomputers) and his GSoC work, we moved ListenBrainz to beta in July after several months worth of intense hacking to revamp core technologies and to make things more stable. In August we published the results of the student Google Summer of Code projects, which capped off a successful season of Code-In and Summer of Code.
Unfortunately the relationship between Quesito and the Foundation deteriorated over the summer and her contract was not renewed. She raised a number of issues with the board of directors, who proceeded to launch an investigation into the situation. This would take through the end of the year to resolve.
In September we also saw more team changes. Roman Tsukanov decided to not renew his contract and to replace him we hired two of our Summer of Code students. We hired both Param Singh (working on ListenBrainz, CritiqueBrainz and AcousticBrainz!) and Sambhav Kothari (aka samj1912) working on Picard and picking up the languishing Solr search project to replace our old search servers.
In September we also held our annual summit in Barcelona, complete with a live stream of the event. We posted the summary of the summit to our blog.
The remainder of the year was fairly quiet and we only had the usual server releases and other community activity.
The high end audio gear manufacturer Naim joined as a Gold supporter in February and the concert promoter AEG Presents joined on the Unicorn tier at end end of march.
In May the audio experience company Qroom joined as a Bronze support and in August another high end audio gear company, Byston, also joined on the Bronze tier.
In 2017 we also had another company join as on the Unicorn tier, but this company wasn’t ready to reveal this new relationship at the time, so we cannot yet mention its name.
When the year started, we were still suffering from the invalid QNAP traffic and we were receiving 2,000 requests/second. A lot of our services hadn’t become stable yet since moving to Hetzner and it took us nearly half the year to get our services fully stable on the new hosting infrastructure. Towards the end of the year as QNAP NAS’ were being updated, the traffic had dropped to 1,300 requests/second. To put that into context, about 265 requests/second were legitimate traffic that we honored; from mid-2017 we were able to respond to 100% valid requests against our web sites and APIs.
The mean response time on MusicBrainz website was between 400 ms and 500 ms, and between 70 ms at the start of the year and 30 ms at end of the year for the web service. While the web service improved greatly, the response time for our web pages was far too slow. Most of these slow response times were caused by our aging search servers and really slow template generation for our web pages.
To address both of these issues, we focused our energy on re-starting work on our new Solr based search system that had been in-progress for several years. We also decided to move our web page rendering to the modern React toolkit, but realized that it would take many months of work to make this transition.
We hit our peak traffic in mid-May when we received 25 mbit/s and transmitted 125 mbits/s. At the end of the year we were receiving 14 mbit/s and transmitting 67 mbit/s and our systems were finally stable after a turbulent year of improving our hosting infrastructure.
In 2017 we earned the following income and had the following expenses:
| Income | |
|---|---|
| Donations - PayPal | $ 6,126.96 |
| Donations - General | $ 32,353.01 |
| Consulting - General | $ 10,900.00 |
| Bank - Interest | $ 62.87 |
| Affiliate - Amazon | $ 245.43 |
| Affiliate - Tagger | $ 4,018.47 |
| Sponsorship - Summit | $ 10,000.00 |
| Support - Bronze | $ 28,067.66 |
| Support - Silver | $ 13,367.64 |
| Support - Gold | $ 19,900.00 |
| Support - Unicorn | $ 249,275.05 |
| Support - Stealth supporter | $ 2,073.51 |
| Support - Development | $ 15,000.00 |
| Exchange Gain or Loss | $ 1,411.96 |
| Total income: | $ 392,802.56 |
| Expenses | |
|---|---|
| Bank - General | $ 569.54 |
| Bank - BBVA | $ 533.70 |
| Bank - PayPal | $ 736.97 |
| Hardware | $ 618.86 |
| Travel | $ 3,332.26 |
| Internet | $ 1,646.78 |
| Marketing | $ 1,144.69 |
| Development - Contractor - Foreign | $ 38,183.61 |
| Officer Salary | $ 82,417.66 |
| Supporter management | $ 17,221.87 |
| Community management | $ 23,369.21 |
| Development - Contractor - US | $ 46,142.91 |
| Customer service | $ 9,877.09 |
| Events | $ 10,689.86 |
| Filing Fees | $ 95.00 |
| Software | $ 383.99 |
| Entertainment | $ 730.00 |
| Books | $ 31.97 |
| Insurance | $ 2,335.00 |
| Accounting | $ 3,625.00 |
| Shipping | $ 14.42 |
| Bank - WePay | $ 5.12 |
| Hosting - Hetzner | $ 21,437.69 |
| Bank - Stripe | $ 298.83 |
| Hosting - Google | $ 4,652.56 |
| Sysadmin | $ 35,403.38 |
| Office Buenaventura Muñoz 15 | $ 1,964.50 |
| Rent Buenaventura Muñoz | $ 9,737.28 |
| Utilities - Buenaventura Muñoz office | $ 2,033.37 |
| Hosting - AWS | $ 1.76 |
| Total expense: | $ 319,234.88 |
| Excess income: | $ 73,567.68 |
Our revenue continued to increase over the course of 2017! While we reduced our hosting costs in 2017, the biggest change was bringing on more supporters and having the supporters who joined late in 2016 start paying us.
In the course of 2017 we spent $618.86 on hardware for team/community needs not related to hosting. In total we spent $26,092.01 on hosting at Hetzner, Google Cloud and Amazon AWS, down from $29,648.95 for 2016. Our overall hosting costs have increased a little as compared to costs from our California hosting, but we are operating more sites with more throughput and much lower response times.
In 2017 we spent $ 84,326.52 on salaries for developers, up from $ 75,253.49 in 2016. We also spent $ 35,403.38 for system administration, $ 23,369.21 for community management, $ 17,221.87 for supporter management, $ 9,877.09 on customer support and $ 82,417.66 on administration. Paying for our contractors remains our biggest cost, as it is customary for most businesses.
In 2016 we had earned $ 254,305.07 from licensing our data and that grew to $ 312,683.86 in 2017. We also earned $ 15,000 from one supporter that sponsored the development of our new JSON data dumps for MusicBrainz. Our income from licensing our data continues to increase due to us taking on more customers, both from changing our business model in 2015 and more supporters needing to use our data.
The silver and gold supporter categories picked up in 2017; also we gave up on the mobile tier in 2017 and instead created a stealth supporter category which is designated for companies that are supporting us, but do not yet have a public product and wish to remain out of the spotlight. While this category gave us more income than the mobile tier, we can see that stealth supporters are also not contributing much to our bottom line.
Our end user donations via PayPal came to $ 6,126.96 which is nearly unchanged from 2016 when they were $ 6,177.44. Direct/large donations added up to $ 32,353.01, including $ 30,000 annual support from Google’s Open Source Programs Office and some cash donations from smaller donors. Google also sponsored our annual summit for $ 5,000 and the sponsorship payment from 2016 hit our account in 2017, thus the financial categories above show $ 10,000 in summit support. A big thank you to Google for supporting our mission and our summits!
Number of editors who made at least one edit in 2017, including accounts that were deleted: 22,381. This represents an improvement over 2016, when it was 20,989 editors. The number of voters who made at least one vote in 2017, including accounts that were deleted was 1,030. This continued to decrease from 1,037 in 2016.
Note: All of the statistics may include editors whose accounts have been deleted in the course of the year.
| Top Editors | |||||
|---|---|---|---|---|---|
| 1 | TheBookkeeper | 718,733 | 13 | Senax | 88,249 |
| 2 | stupidname | 408,904 | 14 | 221bbs | 84,686 |
| 3 | reosarevok | 301,603 | 15 | wcw1966 | 84,251 |
| 4 | ListMyCDs.com | 246,966 | 16 | ProfChris | 77,796 |
| 5 | drsaunde | 130,711 | 17 | rochusw | 74,542 |
| 6 | HibiscusKazeneko | 115,313 | 18 | a23bed | 71,174 |
| 7 | CyberSkull | 114,443 | 19 | vzell | 66,578 |
| 8 | monxton | 111,509 | 20 | loujin | 65,983 |
| 9 | MajorLurker | 109,055 | 21 | silentbird | 64,209 |
| 10 | wonder_al | 93,869 | 22 | MrH | 61,795 |
| 11 | tigerman325 | 92,869 | 23 | pomes27 | 57,813 |
| 12 | fmera | 89,428 | 24 | pmepepnoute | 53,790 |
| Top Voters | |||||
|---|---|---|---|---|---|
| 1 | muddipanda | 73,607 | 13 | monxton | 11,701 |
| 2 | fmera | 68,369 | 14 | TheBookkeeper | 11,162 |
| 3 | otters61 | 49,132 | 15 | drsaunde | 11,077 |
| 4 | chabreyflint | 47,000 | 16 | Senax | 10,388 |
| 5 | jpmoss | 34,092 | 17 | viper666 | 7,839 |
| 6 | 221bbs | 29,697 | 18 | JJLvK | 7,147 |
| 7 | Jazzy Jarilith | 24,481 | 19 | tigerman325 | 5,709 |
| 8 | MrH | 21,340 | 20 | ListMyCDs.com | 5,670 |
| 9 | Alioth | 20,973 | 21 | Sophist | 4,093 |
| 10 | rdswift | 16,808 | 22 | Kid Devine | 3,983 |
| 11 | jesus2099 | 15,239 | 23 | steinbdj | 3,865 |
| 12 | Cheezmo | 13,136 | 24 | reosarevok | 3,516 |
Most of 2017 saw a very rapid growth of new accounts on MusicBrainz and also an increase of noticeable spam in the system. For most of 2017 between 350 and 1,000 new accounts were created each day. The September 18th MusicBrainz server update caused a big drop in new accounts, having it hover just below/around 200 new users a day, which means that up to 800 of the daily new users until October were likely spam accounts. There are still some spam accounts getting created, but for most of MusicBrainz’s history we’ve gotten around 150–200 new accounts per day, so it’s hard to say currently how many of the new accounts are actually spam accounts.
But even if we don’t see as many new spam accounts anymore, we do at least have a big collection of old, dormant spam accounts. To both fight new, incoming spam and clean up all of the old spam, we started planning a new project in 2017: SpamBrainz. The idea for SpamBrainz is that it should analyse user information as well as edits made to the database to see whether they look like spam, and allow humans (MusicBrainz editors with the https://tickets.metabrainz.org/browse/MBS-9480 “spam ninja” denomination) to agree or disagree with this classification. SpamBrainz didn’t see a lot of progress in 2017 beyond the planning of a Summer of Code project for 2018.
At the end of 2017 we had the following servers hosting our sites for us:
This is a total of 19 machines, with each of the machines in our main MetaBrainz cluster running Docker. All of our services are broken down into containers, which allows us to easily start/move/stop services between servers in order to efficiently use all of the machines.
In early 2017 we added two more servers, Williams and Lemmy with the goal of using them for analytics and ListenBrainz respectively. Later in the year Hetzner started offering more capable and cheaper AMD based servers, so we expanded our setup with Burnside, Yehudi, Whetton and Cage to mainly add more MusicBrainz web page capacity.
Zas and all of our engineers continued to work all throughout 2017 to make our sites more robust, respond faster and be more redundant. 2017 marked a vast improvement of for the reliability of all of our sites. This caused the overall sleep for the MetaBrainz engineering team to improve pretty significantly.
Many thanks to our editors, voters, peer reviewers, bug watchers and other members of our community -- without you MetaBrainz would not be what it is today!
We'd like to also thank our developers that pushed out dozens of releases of the site, Picard, client libraries, AcousticBrainz, CritiqueBrainz, ListenBrainz and the Cover Art Archive. All of your work is critical to enabling the MetaBrainz community to do its job.
We'd also like to thank our awesome Board of Directors and all of the donors who contributed money and all of our supporters who make it possible for us to employ a team of open source hackers scattered across the globe.
In particular we'd like to thank Google for its annual support and the support of our summit. These large contributions allow us to carry on with our mission of making MusicBrainz the most comprehensive music encyclopedia out there and pressing on with making our other projects more relevant.
Thank you to everyone who contributed in 2017!