Here at the DCA we manage a lot of material. We have images, folders of documents, ground breaking shovels and other three dimensional objects, digital files, books, and A/V material. In order to manage the material effectively, we need a system that helps us identify, describe, and locate it all.
In 2010 we undertook a project to replace our old collection management system with a new tool that would help us do our job better. This tool is an open source web-based application called CIDER (source code can be found on GitHub.) However, creating the application was only the first step of replacing our old system. We also had to migrate all of the data that existed in the old system to CIDER.
The migration process was no small feat. We had over 600 collections to migrate. Some collections had only a handful of records. Others had thousands of records. We had to standardize, clean up, check the accuracy, and transform all of the existing data before we could import it into CIDER. Once it was in CIDER, we had a complex QA process to ensure that every piece of information was migrated accurately and completely. Each record was touched at least four times before it was considered complete. It took some excellent coordination, motivation, and commitment from those who helped the process go forward and the staff who had to work in an environment where our our data was in two different places at once!
We are happy to announce that the migration and QA process is now complete and we wanted to share some statistics from the process:
- There were nearly 250,000 records migrated as part of this process.
- The first collection was frozen in the old system to prevent changes to the data was on December 20, 2011.
- The QA process was finished on the final collection, 26 months later, on February 24, 2014.
- A quarter million records in 616 collections averages to about 406 records per collection.
- On average, we were able to migrate approximately 2200 records per week.
- Three people worked part-time on the first step of migrating data from one system to another
- Six people worked part-time on the final QA steps.
Time to pull out the tinsel and throw a party!