| Dataset | Description | Number of Samples | Tag Information | Text Information |
|---|---|---|---|---|
| MetaScore-Raw | The raw MuseScore files and metadata scraped from the MuseScore forum | 967K | ✔ | |
| MetaScore-Small | A subset of MetaScore-Full that contains ground truth genre tags | 186K | ✔ | ✔ |
| MetaScore-Full | MetaScore-Raw were missing genre tags are completed by the trained genre tagger. Note that MetaScore-Full is larger than MetaScore-Raw as we also include musical scores without paired metadata | 1.27M | ✔ | ✔ |
| MetaScore-Full-Caps | A subset of MetaScore-Full that contains LLM-generated captions | 623K | ✔ | |
| MetaScore-Small-Caps | A subset of MetaScore-Small with LLM-generated captions | 104K | ✔ |
| Model | Training Dataset | Input Type | Model Size | Training Samples |
|---|---|---|---|---|
| MST-Tags-Small | MetaScore-Small | Tag | 87.36M | 150K |
| MST-Tags | MetaScore-Full | Tag | 87.36M | 901K |
| MST-Text | MetaScore-Full-Caps | Text | 87.44M | 560K |
We show annotation examples in MetaScore in the following table. The first two examples are from genre tagger test set with both true genre tags and inferred genre tags. Example 3 and 4 are from those without true genre tags. Example 5 and 6 don’t have inferred genre tags because they are from genre tagger training set. Note that the following samples are not generated by AI.
| Audio Sample | True Genre | Inferred Genre | Extracted Metadata | LLM-Captions |
|---|---|---|---|---|
| Classical/Traditional | Classical/Traditional | Classical/Traditional; Easy; William Marshall; Public Domain; 1 view | A easy classical music piece composed by William Marshall. This music piece is free for use by anyone for any purpose. | |
| Rock/Metal | Rock/Metal | Rock/Metal; Intermediate; All rights reserved; Piano; 5 comments; 158 favorites; 5862 views | A music piece with a rock/metal vibe. | |
| ✘ | Soundtrack/Stage | Sountrack/Stage; Trombone, Piano | This is a soundtrack with a trombone and piano in it | |
| ✘ | Jazz/Blues | Jazz/Blues; intermediate; Attribution, non-commerical, share alike; Piano; 1 comment, 6 favorites and 462 views | A intermediate piano music piece. This music has jazz/blues vibe. Users must give credit to the creator when using this music. | |
| Rock/Metal | ✘ | Rock/Metal; All rights reserved; Bass; 2 comments; 255 favorites; 5670 views; acaphella; bad romance; cover; lady gaga | A bass music piece with rock/netal vibe. This music piece is a cover of Lady Gaga’s Bad Romance. | |
| Soundtrack/Stage | ✘ | Soundtrack/Stage ; easy; Attribution; Piano; 11 comments; 20 favorites; 767 views; finding paradise; finding paradise piano; finding paradise where are you; finding paradise where are you piano; piano; to the moon; videogame; videogames music; videogames piano; videogames; score; where are you finding paradise; where are you finding paradise piano; where are you piano | This is a easy piano music piece. this music piece is a soundtrack for a videogame |
Note: LLM-captions are generated using true genre tags when these are available. If accurate tags are not available, the captions are generated using inferred genre tags instead.
Settings: In terms of tag-conditioned music generation, we initiate with a “start-of-song” event, then feed relevant tags into either MST-Tags-Small or MST-Tags. This is followed by a “start-of-notes” event to allow the model to begin generating notes. For text-conditioned music generation, we input the text embedding corresponding to the provided text into MST-Text and use a “start-of-song” event to prompt the model to start generating notes.
| Model | Condition | Audio Sample |
|---|---|---|
| MST-Tags-Small | Classical/Traditional, Hard, ludwig van beethoven, Piano | |
| MST-Tags | Classical/Traditional, Hard, ludwig van beethoven, Piano | |
| MST-Text | This is an easy classical piano piece composed by wolfgang amadeus mozart. |
| Model | Condition | Audio Sample |
|---|---|---|
| MST-Tags-Small | Rock/Metal, Electric-guitar, Electric-bass, Piano | |
| MST-Tags | Rock/Metal, Electric-guitar, Electric-bass, Piano | |
| MST-Text | This is a rock or metal music. It has piano, guitar and bass within it. |
| Model | Condition | Audio Sample |
|---|---|---|
| MST-Tags-Small | Drum, Synth_Bass | |
| MST-Tags | Drum, Contrabass, Piccolo | |
| MST-Text | This music has drum within it |
The following music pieces are generated by MST-Tags-Small given tag conditions.
| Condition | Audio Sample |
|---|---|
| Easy, Piano | |
| Advanced, Piano | |
| Jazz/Blues, Piano, tenor-saxophone | |
| Classical/Traditional, Wolfgang Amadeus Mozart, Easy, Piano | |
| Electronic/Dance |
The following music pieces are generated by MST-Tags given tag conditions.
| Condition | Audio Sample |
|---|---|
| Easy, Piano | |
| Advanced, Piano | |
| Classical/Traditional, Wolfgang Amadeus Mozart, Easy, Piano | |
| Electronic/Dance | |
| Jazz/Blues |
The following music pieces are generated by MST-Text or BART-base Text-to-Music given text prompts. In each row, we present one example generated with MST-Text and another with BART-base Text-to-Music, both generated from the same text prompt.
| Text Prompts | Audio Sample | Audio Sample |
|---|---|---|
| Generated with MST-Text | Generated with BART-base | |
| A classical music piece composed by Wolfgang Amadeus Mozart. | ||
| This is a soundtrack with electronic/dance vibe. | ||
| A classical music piece with Jazz/Blues vibe. This music piece is influenced by George Gershwin. | ||
| This is an easy folk/country music piece. | ||
| This is a intermediate classical music piece. This music piece has piano in it. |