Open Source Research #
Project Outcomes #
-
Open Source Research finished successfully in September 2022 after 2 years of work. We built a neural audio synthesiser controlled via speech commands as planned.
-
You can try out the system installing it from this GitHub repo or play around with the generation module on this webapp.
-
We published our research at the 3rd Conference on AI Music Creativity 2022. The title of the paper is “From Words to Sound: Neural Audio Synthesis of Guitar Sounds with Timbral Descriptors”.
-
You can check out demo samples generated with the system in the paper companion website."
Research Project #
-
OpenSource Research is an experimental project which aims to create a piece of collaborative research in the field of AI music.
-
The members of The Sound of AI (youtube channel) community collaborate to carry out research, following the philosophy and practices of open source software and open research.
-
The selected topic is about creating a Sampler powered by a NeuralNet synthesizer that generates guitar sounds with text descriptors and voice inputs, does it sound complex ? let’s look at it simply with this word collection :
- The final paper titled From Words to Sound: Neural Audio Synthesis of Guitar Sounds with Timbral Descriptors has been submitted and presented at the 2022 Conference on AI Music Creativity. You can read the paper at the link above or visit the companion website to check out some examples of the generated sounds.
Kickoff video #
Get Involved #
steps details in How can I get involved? section
Slack Channel #
All discussions take place here :
Discord server #
Voice meetings and small groups presentations
Github #
organization
Meetings and presentations
Google Drive #
Project Details #
Project goals #
The project has a number of research-oriented goals:
- Develop an application, methodology, or algorithm that advances knowledge in the field of AI music. Contributors will choose the specific topic to work on together at the onset of the project.
- Develop a code base with the new application / algorithm that other researchers can access openly.
- Write up research results in an academic paper.
- Submit paper at the 2022 Conference on AI Music Creativity.
Who can get involved? #
Everyone in The Sound of AI community can contribute to OpenSource Research, regardless of your skillset.
In this research project, there will be many different tasks, such as coding, testing algorithms, evaluating models, writing up research, proofreading, managing the project, ….
Like in an open source coding project, members with different skills can contribute to different tasks, depending on their expertise and their time availability.
There are only two requirements to participate in the project. You should have an interest in AI music. You should follow through with what you commit to work on.
How can I contribute? #
Depending on your skillset you can contribute to one or more of these activities:
- Brainstorming ideas (topic)
- Brainstorming technical / methodological solutions
- Reading and summarising papers related to the topic of interest
- Writing code
- Testing code
- Writing documentation
- Evaluating models by running in-silico experiments
- Designing / running user studies to evaluate models
- Searching for datasets online / building a dataset
- Cleaning / preprocessing data
- Managing the project
- Writing up research
- …
What do I get by participating? #
For members less experienced in research, OpenSource Research is a great opportunity to:
- Learn how to write research papers
- Learn how to carry out research from scratch and manage research projects
- Improve your research skills by learning from more experienced researchers
- Network with other researchers and practitioners
For more experienced members, this project is an opportunity to:
- Improve project management skills
- Give back to the community, by training less experienced researchers
- Network with other researchers
- Experiment a new way of doing research
For everyone, OpenSource Research is an opportunity to:
- Be part of a unique research experiment that may have a transformative impact on the wider academic community
- Work on a super cool research topic
- Be part of an amazing group of people
- Have an impact on the field of AI music
How can I get involved? #
To get involved with OpenSource Research, follow these steps:
- Fill up this survey.
- Sign up to The Sound of AI Slack workspace, if you aren’t already a member of the community.
- If you’re a member of The Sound of AI Slack, subscribe to the #open-source-research channel. Communication about the project will happen there.
- Once you have submitted the survey, Valerio will send you an email with further instructions and a link to sign-up to The Sound of AI Slack.
- Valerio will add you to all the services used to carry out research (Google Drive, Trello, etc) – see “Project tools” section for more info.
Please note that Valerio will send out emails / add members to services in batches, once per week.
Project tools #
OpenSource Research is a complex project. It will use a number of different online tools to enable contributors to communicate, collaborate, and manage the work.
Communication #
The Sound of AI Slack #
The Sound of AI Slack workspace is the main venue where conversation among contributors happens. Here members can find announcements, talk to each other, and share resources.
All the conversations regarding the project will happen in the #open-source-research channel.
Discord #
Discord will be used to host video conference calls of small research groups in parallel.
OpenSource Research mailing list #
The mailing list will be used to share general news, and let contributors know about upcoming meetups.
GoogleMeet #
GoogleMeet will be used to host conference calls for brainstorming and catch-up sessions.
Collaboration #
Google Drive #
A dedicated folder on Google Drive will be used to store materials relevant for the research project (e.g., papers, data).
Google Doc #
Google Doc will be used to collaborate on the write-up of the paper. Members will be provided with “suggesting” privileges. Valerio (or lead researchers) will integrate text written by contributors.
GitHub #
A Github organization will be used to collaborate on the code developed for the project.
Project management #
Trello #
Trello is an easy-to-use online project management tool that follows the Kanban system. Trello will help organise the work in modular tasks, keep track of the status of different tasks, and of who is working on what. Here is a tutorial that gets you started with Trello.
Project roles #
Project leader #
Valerio will coordinate the research project, and will choose the overall direction of the project based on the input coming from contributors.
Project managers #
There are two project managers: Fernando Garcia and Wassim Filali. Their role is to coordinate activities and contributors. They also plan workflows, goals, and schedule for different project phases
Research coordinators #
Research coordinators are in charge of a Research Group (RG). A research group is a team of 5-8 contributors focused on one research deliverable. Research coordinators plan work and dedlines for a RG following the overall goals set by project leader and project managers. They act as scrum masters for a RG.
Research co-coordinators (CoCos) #
CoCos provide logistical support to research coordinators.
Contributors #
Contributors will steer all the aspects of the research project (e.g., topic selection, solution design / implementation, evaluation, etc). They will be involved at all levels, from ideation to implementation.
Project steps #
The project will be organised in a number of steps. Some of the steps may run in parallel and feed back into each other.
It is worth noting that these are general indications, which may be re-adjusted based on the needs and specificity of the chosen research topic.
1. Preparation #
In this phase, contributors will get familiar with the AI Music Creativity space, project managers will be identified, and contributors can apply as research coordinators and co-coordinators.
2. Topic selection #
In this phase, contributors will choose a topic that is in line with the scope of The 2021 Conference on AI Music Creativity.
3. Literature review #
During this step, contributors will gather, analyse, and discuss papers from the scientific literature that are related to the topic of interest. They will outline existing weaknesses and limitations in current solutions and suggest what is missing.
4. Solution design #
In this step, contributors will have brainstorming sessions in order to come up with tentative solutions that address the chosen topic / problem.
5. Solution implementation #
In this phase, contributors will develop the solution. This will likely be in the form of code.
6. Evaluation #
During this step, members will evaluate the validity of the proposed solution. Depending on the chosen topic, this will happen in different ways. For example, the proposed model(s) can be tested against other base-line solutions, or they can be evaluated using user studies.
7. Write up #
In this phase, contributors will write up the research they carried out.
8. Submission #
The paper has been submitted to The 2022 Conference on AI Music Creativity.
9. Revisions #
In this phase, contributors will address the concerns of the reviewers and resubmit the improved paper.
Timeline #
Even though it is difficult to come up with a reliable timeline at this point, I will propose a tentative one:
- Preparation - 2 weeks
- Topic selection - 2 weeks
- Literature review - 4 weeks
- Solution design - 3 weeks
- Solution implementation - 6 weeks
- Evaluation - 4 weeks
- Write up - 4 weeks
About #
License and authorship #
All the work produced (e.g., code, databases, algorithms) will be distributed using the MIT License.
The authors of the paper will be Valerio and The Sound of AI community.
Netiquette #
- Be always respectful
- Be kind
- Be supportive
- Try to give more than take
Questions about research processes #
The project can indirectly provide insights into questions about research processes:
- Is there another viable way to carry out research that avoids the pitfalls of how research is traditionally carried out (e.g., focus on the number of papers published, rather than research quality)?
- Can open / crowdsourced research produce high-quality output?
- Can contributors with different expertise collaborate effectively towards a common research goal?