Behind the Scenes: What it takes to run ESMARConf

Event Fund supports more inclusive and accessible events in data science in order to lower barriers to participation in science and facilitate new collaborative relationships. Part of supporting emerging data science leaders to run good data science events includes developing a knowledge base for what it takes to run outstanding and creative events. In this post–part of an ongoing series that focuses on the organizing processes, tools, and practices used to run successful Event Fund-sponsored Open Science events–we learn about ESMARConf and how they have used automation to try to streamline a lot of the behind-the-scenes work. Through this blog series, Event Fund seeks to highlight the processes and practices that undergird more inclusive and accessible data science event programming.

Evidence Synthesis and Meta-Analysis in R Conference (ESMARConf) is a free annual online conference coordinated by the Evidence Synthesis Hackathon aiming to connect tool developers and users to raise awareness of existing resources and support the development of novel needs-based tools to facilitate evidence synthesis and meta-analysis in R. The conference aims to build an interdisciplinary community of practice, where individuals can exchange methodologies and tools with one another.

Following two successful conferences held in 2021 and 2022, we are now in the final stages of planning a third conference in 2023, continuing the special focus on inclusivity and accessibility. Register to attend the upcoming ESMARConf 2023 (27 - 31 March) here!

Timeline

• June 2020
  Active members of the Evidence Synthesis Hackathon began planning for the first ESMARConf.
• January 2021
  The first ESMARConf was held from January 21 - 22.
• February 2022
  The second ESMARConf was held from February 21 - 24. ESMARConf2022 consisted of twenty-eight presentations, three panel discussions, and six workshops. All conference content is available for viewing on YouTube.
• March 2023
  The third ESMARConf is due to take place March 27 - 31. It will consist of around 28 presentations, more than 55 20-minute tutorials, 10 interactive 2-hour workshops, and 9 1-hour panel discussions - it is the biggest and most ambitious ESMARConf yet!

Sociotechnical Infrastructure

The term “sociotechnical infrastructure” acknowledges how complex and interconnected systems of human relations, technical objects and tools, and underlying processes shape and are shaped by each other. In this section, we highlight the systems that event organizers established in order to thoughtfully organize and run their events.

One of our key priorities was to create a conference which was accessible to all who were interested. This meant that we aimed to promote inclusivity at all stages of the conference.

Communication

We initially used Slack, email, and WhatsApp to communicate with one another. In planning ESMARConf2022, we started communicating primarily through GitHub, due to its increased transparency and organization. We have also started to use R to develop an automated system which would send monthly emails, detailing conference dates and noting important deadlines. R was also used to automatically upload files to Zenodo, and is planned to schedule the release of Tweets and other repetitive tasks in the future.

Raising awareness

Social media has played a critical role in raising awareness of the events, with Twitter predominantly being used to advertise the conference. Besides using social media, the mailing lists from previous attendees of ESMARConf, along with lists from other interested organizations, were contacted. We have now built a large audience of over a1000 people who have subscribed to our mailing list.

Conference Software

The conference uses free or low-cost software in order to reduce costs of running the conference, allowing the event to be completely free for participants. Presenters are asked to pre-record individual talks, which are then uploaded to YouTube and subtitled - automatically at first, then manually verified, which allows for relatively accurate auto-translation. Participants interested in attending workshops can register directly through Zoom. Workshops are held over Zoom and while typically limited to fifty participants, they are additionally livestreamed to YouTube, enabling us to reach a wider audience.

Panel discussions are held over closed Zoom calls, which are also livestreamed to Youtube. This format means that viewers can watch content at their own pace. Q&A takes place primarily through Twitter, with registered participants also communicating through a dedicated Slack channel.

In 2023, more of the core content will be published on YouTube as ‘premieres’ - pre-recorded talks that are assembled into full sessions and scheduled for release at a given time. This means less pressure on us as organizers to run live-streams.

Reaching new audiences

ESMARConf2022 and ESMARConf2023 were geared towards anybody interested in conducting or learning more about evidence synthesis. This included coders producing evidence synthesis tools who wished to collaborate with one another, R users who wished to conduct evidence synthesis in R and learn about available tools, and non-R users with experience in evidence synthesis wishing to use free tools with non-coding user interfaces.

Increasing accessibility

The format of the conference itself removes barriers that typically arise in traditional settings. By organizing the conference as an exclusively online event, we hope to increase the number of participants from traditionally underrepresented regions and countries. Holding the conference online and recording all live content allows participants to bypass barriers like timing and geographic location - ESMARConf2022 had participants from 86 countries.

Measures are taken so language is not a barrier to accessing the conference. The Event Fund grant has been used to fund a team of transcribers and editors, who manually verified automatic transcripts of all pre-recorded individual talks before the start of the conferences. Users can then automatically translate closed captions into any language provided by YouTube, including more than 155 languages.

We have been able to create a more inclusive conference by removing any obligatory conference fees, although voluntary donations are encouraged, as funding is needed to cover the costs of verification of subtitles, provision of bursaries, and small software costs- primarily, a Zoom subscription to support livestreaming to YouTube. The Open Collective events page has been used for ticketing. The platform enables us to easily set up events and facilitates the transfer of funds to a collective budget when individuals make a voluntary donation. In addition to being completely free, financial support is provided as individual bursaries in cases where individuals would have otherwise been unable to attend the event.

Finding time to join an online conference can be challenging for parents or those supporting people with disabilities. Resource constraints pose another barrier to participation, particularly for those in low- and middle-income countries. In order to address these barriers, in 2022 bursaries of up to 100 USD were provided. These bursaries could be utilized in a number of ways to break down barriers to participation. Those with caregiving responsibilities could hire an alternative caregiver or babysitter, buy ready meals to offset the time taken away from meal prep, or rent a quiet space for concentration. Bursaries could be used to cover the cost of mobile data, renting a laptop or headphones, or the commute to a space with stable internet connection. In 2023, bursaries of 50 USD for resource constraints and 100 USD for caregiving have been offered. Although being widely advertised on the conference website and social media, these bursaries have never been fully subscribed, so our bursaries budget has twice rolled over to the following year.

“I think the most amazing bursary story was for someone asking to hire a diesel generator and mobile internet from Nigeria…” reported Neal Haddaway, conference organizer.

Twenty-six bursaries were awarded in total in 2022. We received fewer applications than expected, but those who did receive bursaries reported that the support had a great impact on their ability to participate.

What’s next?

We have reflected on the development of the conference in a paper published in 2022. ESMARConf2023 is due to take place March 27 - 31 2023, with another event already being planned for 2024.

Feedback from participants in 2022 noted that many of the presentations didn’t include a visual component and could be easily listened to, rather than ‘viewed’. As a result, we are planning to make the most of this content by converting 2022 talks into audio-only podcasts to be released later in the year, further broadening the potential audience.

In the future, we also hope to be able to let those giving pre-recorded talks present in the language of their choice and to provide subtitles which can then be translated into the viewer’s preferred language. We also plan to more clearly separate pre-recorded sessions from discussion sessions, in order to better organize efficient presentations and active discussions sessions.

In 2023 we relied on a voluntary registration payment system, with the aim of giving the conference a sustainable long-term plan for funding on top of support from grants and foundations. By introducing an option to voluntarily buy a ticket within the range of 5-200 USD, individuals with the capacity to pay can ensure that the conference remains accessible to anybody interested in attending. So far we have raised almost 3,000 USD in 2023, which will go to support next year’s activities.

ESMARConf2022 spent less than expected, and was granted a no-cost extension allowing organizers to use these funds to support future ESMARConfs.

We have received over 4 voluntary organization donations, and have therefore been able to secure 10,000 USD for ESMARConf2024 and 2025.

The Evidence Synthesis Hackathon is now looking for new organizing team members interested in helping to coordinate ESMARConf2024. Contact Neal Haddaway for more information. And if you are looking to register to attend the ESMARConf 2023 (27-31 March), you can do so here!

This post is part of an ongoing series that focuses on the organizing processes, tools, and practices used to run successful Event Fund-sponsored Open Science events. Through this series, we seek to highlight the processes that undergird more inclusive and accessible open data science event programming.

Photo by Jack Cohen on Unsplash