The objective of the International Music Information Retrieval Systems Evaluation Laboratory project (IMIRSEL) is the establishment of the necessary resources for the scientifically valid development and evaluation of emerging Music Information Retrieval (MIR) and Music Digital Library (MDL) techniques and technologies. Part of the project is the creation of secure, yet accessible, large-scale collections of music materials in a variety of audio, symbolic and metadata forms. These collections, when coupled with a set of standardized experimental tasks and standardized evaluation metrics, will allow members of the international MIR/MDL research community to participate in TREC-like evaluation "contests" so they can scientifically compare and contrast their various approaches to making the world's vast store of musical heritage materials ever more available.