Data Curation- The Art of Selecting and Appraising Data

 Here is the truth nobody likes to here: not all data is worth saving. We live in a time that generates information at a breathtaking pace, yet the instinct to keep everything can do more hard than good. Without thoughtful selection, repositories become cluttered warehouses where valuable datasets get buried under mountains of noise. that is why selection and appraisal sit at the very heart of good data curation.


What Do Selection and Appraisal Actually Mean?

In the DCC Curation Lifecycle Model, appraisal is described as the process of evaluating data to determine what merits long-term curation and preservation. Selection is the decision that follows: which datasets stay, and which ones do not. Together, they act as a quality filter, ensuring that the data we invest in preserving is genuinely worth the effort.

Think of it like editing a book. A first draft has raw material, some brilliant and some not. An editor does not keep every sentence out of loyalty to the writer. They keep what serves the reader. Appraisal works the same way, except the reader is every future researcher who might depend on that data.


DCC Curation Lifecycle Model showing appraisal and selection as key activities


The DCC Curation Lifecycle Model- appraisal and selection are central activities in the cycle.

Why We Cannot Keep Everything

Storage is cheap, but curation is not. Creating proper metadata, ensuring file formats remain accessible, and maintaining documentation all cost time and expertise. The Digital curation Centre makes a compelling point: reducing the quality of data we maintain actually improves the quality of what we preserve. Less clutter means better metadata, more reliable verification, and stronger trust in the collection.

Deciding what not to keep is just as important as deciding what to keep. Good curation requires the courage to let go.

What Makes Data Worth Keeping?

There is no single checklist that fits every discipline, but common criteria tend to surface across the literature. Tallman and Work developed a practical framework rooted in traditional collection development principles, asking questions about a dataset's uniqueness, its potential for reuse, its technical sustainability, and its alignment with an institution's mission.

Cornell University Library, for instance, runs each submission through a curation review that includes file inventory, risk assessment, and appraisal before anything goes live. It is a careful, human process, not an automated sorting machine.

The Human Judgment Behind It All

What I find most compelling about appraisal is that it cannot be fully automated. Algorithms can flag duplicates or outdated formats, but deciding whether a dataset holds long-term scientific or cultural significance requires human judgment. It requires understanding context, recognizing potential, and sometimes making difficult calls about what to let go.

In that sense, selection and appraisal are not cold bureaucratic steps. They are acts of stewardship. Every time a curator decides a dataset is worth preserving, they are making a quiet promise: this matters, and we will take care of it.

FAIR Data Principles- Findable, Accessible, Interoperable, Reusable






References

  1. Whyte, A. & Wilson, A. (2010). How to Appraise and Select Research Data for Curation. Digital Curation Centre.
  2. Digital Curation Centre. Appraisal and Selection. DCC Briefing Papers.
  3. Tallman, N. & Work, L. (2022). Appraisal and Selection for Digital Preservation. Penn State.
  4. Cornell University Library. Data Curation Service. Cornell Data Services.
  5. Zhang, Y. (2014). Appraisal and Selection for Digital Curation. ResearchGate.
  6. Lee, C.A. & Tibbo, H.R. (2011). Digital Curation and Trusted Repositories. Research Data Curation Bibliography.


Comments

Post a Comment

Popular posts from this blog

Why Data Curation Matters More Than You Think

Let me paint you a picture. A reseacher spends two years collecting data, publishes a groundbreaking paper, and then moves on. Five years later, someone tries to build on that work. They find the dataset, download it, and open it only to discover unlabeled colomns, missing documentation, and file formats nobody usess anymore. all that effort, quietly slipping into irrelevance. This  happens far more often that we like to admit, and it is exactly why data curation deserves our attention. what data curation really means Data curation is the ongoing practice of collecting, organizing, cleaning, documenting, and preserving data so it remains useful over time. The goal is to make date FAIR  (Findable, accessible, interoperable, and reusable). It sounds technical, but at its heart, it is about respect: respect for the work that produced the data, and respect for the people who might need it next. A survey published in PLOS ONE found that 97% of researchers who used curation service...
Read more

Data Curation Preservation Issues: Organizational Challenges in preserving Digital Information

Image
In today's digital environment, organizations generate and manage vast amounts of information that must remain accessible and reliable over time. Data curation refers to the active management, preservation, and enhancement of digital assets throughout their lifecycle to ensure their long-term usability (Harvey, 2018). While technological threats often dominate discussions about digital preservation, organizational issues are equally significant and can greatly affect the success of preservation initiatives. One of the most common organizational challenges is the lack of a comprehensive digital preservation strategy . Many organizations create and store digital records without establishing clear policies, procedures, and responsibilities for long-term preservation. According to the Digital Preservation Coalition (2024), the absence of formal preservation frameworks often leads to inconsistent practices, making digital materials vulnerable to loss, corruption, or inaccessibility. Ins...
Read more