The Question of Standards

1.1  Introduction

The flow of incoming new astronomical catalogues and data sets is ever increasing; and since 1993, large tables published in some astronomical journals are also available as electronic files, due to the agreement with the A&A Editors (See Editorial note of A&A 266, E1), and to the development of the electronic publishing for the AAS journals.

The question of a uniform description is raised as soon as we expect to use any of the catalogues in a transparent way, e.g. to ingest the data into a data-base, to load a table of a published paper into our image processing system, etc...

1.2  FITS: why

FITS format (Flexible Image Transport Format, see e.g. the FITS Support Office Home Page) is apparently the most popular data format for the exchange of computer data in our discipline, and is endorsed by IAU. FITS for coding images is widely available (e.g. in the popular xv image displayer), and the FITS way of specifying the position and the geometry of the field of view on the sky (the World Coordinate System or WDS) allows to combine images of the sky and data coming from catalogues in tools like Aladin. FITS defines also a format for tabular data.

1.3  FITS: why not

FITS is well accepted as a transport format; it apparently is also being commonly used for data archiving (e.g. ESO, OGIP) of image or spectral data. However, as far as tabular data is concerned, we prefer not to store the data in any of the two FITS formats defined for tabular data, mainly because

1.4  Proposed solution

We do not want to keep several versions of the same file: not only it would eat up a lot of disk space, but mainly the existence of astronomical catalogues in several data format standards would inevitably generate conflicts and incompatibilities.

In order to be able to use the various tools, we therefore chose to keep the astronomical catalogues as plain ascii files, and to store the description of the catalogue as a separate ascii file. This description file includes all the necessary information related to the catalogue: author(s), reference(s) of the related published papers, brief summary, scientific keys, caption and accurate description of each table of the catalogue.

This description file — the key of the catalogue — is especially important: it must be easily readable by a human eye, and simultaneously contain all the information required to achieve the conversion into other formats if required: for instance, the transformation of tabular data files into FITS can be done automatically from the CDS FTP server; and a client/server prototype, described in the cdsclient package, has been developed for remote copies of the astronomical catalogues in a few formats, including FITS. This description file is also used for checking purposes: a standalone program named anafile, is used in this context.

The standard discussed in this document mainly addresses the tabular data, but some catalogs may include data in a non-tabular form, like maps, images, video sequences, etc... Such non-tabular data should be described in the description file on the file level.

The standards proposed here are developed in the following chapters. Some points in the proposed standards are not completely fixed and possible alternatives are indicated by a note.

1.5  History of this document

The first version of this document was published as the paper Adopted standards for catalogues at CDS published in the Bulletin d'Information du Centre de Données astronomiques de Strasbourg (BICDS), (Ochsenbein F., 1994, BICDS 44, 19).

Version 1.4 of the document was dated 12 September 1994, and resulted from discussion with ADC colleagues, mainly N. Paul Kuin. Common conventions concerned the standard filename conventions — ReadMe as the description file, extensions .dat for the data files — and the basic label definitions.

Version 1.5 is dated 12 June 1996, and contained a few more conventions on file names and label definitions.

Version 2.0 is dated February 2000, and resulted from discussions with G. Schwarz (AAS, Tucson)

1.6  Related Documents

The following related documents are available on the network:

/srv/httpd/Pages/doc/catstd-1.htx