Data Documentation and Metadata
Data documentation will ensure that your data will be understood and interpreted by any user. It will explain how your data was created, what the context is for the data, structure of the data and its contents, and any manipulations that have been done to the data. A great resource is Easy data management: add a README.txt to your project folders.
What's important to document?
- Context of data collection
- Data collection methodology
- Structure and organization of data files
- Data validation and quality assurance
- Data manipulations through data analysis from raw data
- Data confidentiality, access and use conditions
- Variable names and descriptions
- Definition of codes and classification schemes
- Codes of, and reasons for, missing values
- Definitions of specialty terminology and acronyms
- Algorithms used to transform data
- File format and software used
There are a variety of metadata standards, usually for a particular file format or discipline. Some examples include the following:
- Astronomy Visualization Metadata
- Darwin Core
- Data Documentation Initiative (DDI) to document numeric data files
- Dublin Core, a general purpose metadata standard
- ISO 19115 or FGDC's Content Standard for Digital Geospatial Metadata for geospatial data
- Ecological Metadata Language
The UA Library can help you select the most appropriate metadata standard to use. Contact Chris Kollen, Data Curation Librarian.
When creating metadata, a best practice is to use controlled vocabulary, standard terminology for your discipline. Using an accepted standard, controlled vocabulary or an authority list will help in the retrieving and indexing of your data.
Consider keeping metadata records in a spreadsheet, CSV file or tab-delimited file. Additional information needed to interpret the metadata, such as explanations of variable, codes, acronyms or abbreviations, or algorithms used, should be included as accompanying documentation.
In general, the University of Arizona Libraries suggests the following metadata elements:
|Title||Name of the project or collection of datasets|
|Creator||Names and institutions of the people who created the data|
|Date||Key dates associated with the data, such as dates covered by the data or date of creation|
|Description||Description of the resource|
|Keywords or Subjects||Keywords or subjects describing the content of the data|
|Identifier||Unique number or alphanumeric string used to identify the data|
|Coverage (if applicable)||Geographic coverage|
|Language||Language of the resource|
|Publisher||Entity responsible for making the dataset available|
|Funding Agencies||Organization or agency who funded the research|
|Access restrictions||Where and how your data can be accessed by other researchers|
|Format||What format your data is in|