Preface

MIRA as in the current version is still under development. Parts of its capabilities depend on the work done for other program libraries (CLASS90, TELCAL, PAKO). Comments and suggestions from users are welcome, and will be a valuable tool to further improve MIRA.

MIRA in its current version is exclusively used to reduce data written in the new IMBFITS raw data format. For old raw data, the old data processing software (OTFCAL) will be kept and maintained.

As soon as MIRA is fully integrated into as the standard heterodyne data processing software for the new control system, it will further evolve to provide tools for imaging with heterodyne receiver arrays.

This is a summary of recent changes in MIRA:

 


Version 1.1

• For online data processing, a prototypical procedure mira_pro:reduce.mira has been commited to the CVS.

• New default for MIRA\CAL: the calibration parameters of all frontend-backend combinations are computed.

• Output message formats for MIRA\SCAN changed.

• Calibration parameters are already computed and issued when a chopper-wheel calibration is loaded. For spectrometers with several basebands, mean values are given (for basebands belonging to the same spectrum).

• Command syntax for MIRA\VIEW changed to
  MIRA\VIEW [ifb] [/CAL] [/GAINS] [/PHASES] [/SIGNAL] [/MAP]
  Useful defaults are provided.

• Command syntax for MIRA\SOLVE [pointing|focus] [ifb] changed to MIRA\SOLVE [ifb]. Whether a pointing is to be solved or a focus is automatically determined from the observing procedure of the current scan.

• MIRA commands using the SIC macros in mira_pro:p_plot_*.mira can now be retrieved from SIC's command stack (concerns MIRA\SOLVE and MIRA\VIEW).

• MIRA\SOLVE: for pointings, the execution can be halted after the subscan fits with QUIT.

• MIRA command FIND has a new option FIND /NEW (for online data processing). Only new data are put into the index list. If no new data are found, the index list remains unchanged.

• Lists of MIRA commands FIND and LIST: by default, a "short" list is provided (without frontend names and spectral lines). The previous "long" list can be retrieved with the new option LIST /LONG. The telescope name is suppressed in any case. Note that the scan number is now listed in the first column, and the observation number in the last one.

• The information is the current index list can now be retrieved from the new SIC structure list (type \MIRA\VARIABLE LIST r).

• Support of old raw data abandoned.

 


Version 1.2

• Default for MIRA command CAL is now the first frontend-backend combination. If all frontend-backend combinations are to be calibrated, the argument ALL has to be specified, i.e. CAL ALL [/...options].

• The option /OFF of MIRA command CAL now offers several modi for the subtraction of the reference signal in OTF maps with total-power mode. The reference signal is either a mean of all off-measurements (option NONE, the default), an unweighted mean of the reference measurements taken before and after the on-measurement, and weighted means (weighting by total power or elapsed time). See the MIRA language internal help for more details.

• MIRA command WRITE, option /FEBE has a new argument: MIRA\WRITE number /FEBE ALL writes all frontend-backend combinations into the CLASS output file, starting with the observation identification number (if specified, otherwise successively increased by default).

 


Version 1.3

• MIRA stability is improved. If, for a given scan, the imbFits file of one of the backends is faulty, this backend will be ignored in the index list (and a warning message is issued).

• The new imbFits keywords XOFFSETi and YOFFSETi (with to , where is the number of offsets from the catalogue position) are accomodated in MIRA's SCAN%DATA section.The resulting offset is in SCAN%HEADER.

• The SIC macro reduce.mira now becomes reduceCont.mira and reduceSpec.mira. See section 2.1 for details.

• New option /ZOOM for command MIRA\VIEW. First the data is fully shown, and the interactive cursor is called to define a rectangle to be zoomed in. The axis labels are correspondingly updated.

• MIRA\VIEW /MAP /ZOOM can be used to plot a subset of data. In the subset display, the range of ON positions covered by the subscan(s) is indicated.

• MIRA\CAL, option /OFF: the default weight mode for on-the-fly maps is now AVER (taking an average of all available off-source subscans). Option NONE does not remove any off-source signal, but concatenates the spectral basebands.

• MIRA's SIC variable ARRAYi\%DATAj\%ISWITCH (with i frontend-backend number, j baseband number) now becomes an array of character strings (instead of integer flags). See section 5 for details.

 


Version 1.4

• The header information provided by MIRA's command VIEW has been extended.

• MIRA\CAL: if the setup of the backends used for an observation is not strictly the same as for the previous (or, for offline data processing, the subsequent) calibration, MIRA complained about an inconsistent calibration. In version 1.4, MIRA looks whether a calibration is available for exactly the same spectral charactersistics. If so, the calibration is applied, even if the number of the backend part has changed. If no calibration with the same spectral characteristics is available, MIRA issues a warning, and flags those data as uncalibrated.

• Ambigeous scan numbers: it may happen that a raw data directory contains data from different observing dates, but with the same scan numbers. Until now, the only way out was to preselect the data in the index list, using FIND /OBSERVED. Otherwise, MIRA entered into a state of confusion and crashed when such a scan was read with MIRA\SCAN. In version 1.4, this is not so anymore: in case of ambigous scan numbers, the user is asked to enter the desired observing date (if nothing is entered, MIRA\SCAN returns to the command prompt, allowing to opt for the solution using FIND /OBSERVED.

• The pause between pointing solutions by subscans and coadded subscans, respectively, makes automatic online data processing impossible, since MIRA waits for the use to type continue. So, the pause was switched off in version 1.3. In version 1.4, it can be activated using MIRA's logical flag doPause (yes or .true. for interactive offline MIRA sessions, if the user wants to have a close look at the pointing subscans).

• Sometimes there are still spikes in the pointings (although usually lower than or comparable to the signal). MIRA's despiking routine works best after calibration, and the spikes get attributed a blanking value. Therefore, the MIRA\SOLVE command has been upgraded to correctly handle blanked values for either backend counts or equivalent temperature scales.

• slight syntax change for MIRA\VIEW /PHASE idPhase irec. idPhase can be one of the character strings ON, OFF, LOAD, SKY, HOT, COLD, FLO, FHI. The specified phase is averaged across the whole scan, unless a special record is specified with irec.

• If a backend part is disconnected, keeping the others, MIRA will work even if there will be a "hole" in the associated frontend-backend numbers (e.g. using VESPA parts 1, 3 and 4). Not yet tested for more than one backend part lacking.

 


Version 1.5

• The procedure for online data processing is now called onlineDataProcessing.mira. It can be called by
  mira.exe @onlineDataProcessing projId scan obsDate calScan obsDateCal
  where projId is the Pico Veleta project id (e.g. 203-04), scan the scan number to be reduced, obsDate the corresponding observing date (format yyymmdd), calScan the scan number of the calibration scan (may be 0 if no calibration is to be done), and obsDateCal the observing date of the calibration scan (can be different from obsDate). The MIRA SIC procedures reduceSpec.mira and reduceCont.mira are obsolete.

• MIRA\SCAN has a new option /TRACKING trackingError. If specified, all backend data with azimuth or elevation tracking errors larger than trackingError are flagged, i.e. attributed the blanking value for raw data.

• In VESPA polarimetry mode, the fourth subscan of the paKo command CAL /GRID is plotted in blue.

 


Version 1.6

• MIRA commands VIEW, CAL and WRITE have new options for HERA pixels. See online help for details.

• MIRA command FIND is too slow for online data processing if used for large data archives. There are two solutions: either use SCAN without a previous FIND, or at least one of the following options: FIND /OBSERVED yyyy-mm-dd, /SCAN, /BACKEND (FIND /SCAN is fastest).

• You do not need to use FIND to load a scan ! MIRA command SCAN loads a scan, even if no index list exists. In case the scan number is ambigeous, MIRA takes by default the last observing date found (otherwise you have to use FIND /OBSERVED). See the online help for details.

• There is a new utility procedure, odp.mira. It essentially does the same job as the online data processing triggered by the NCS coordinator, but is more interactive. Use: @ odp iRef idScan, where iRef is the frontend-backend combination to be used for pointings and focus solutions, and idScan is the scan to be reduced. The output CLASS file is called spectra.30m, in order to avoid confusion with spectraOdp.30m (produced by the coordinator).

• Large data sets with too short a dump time (e.g. frequency switched on-the-fly maps) can be compressed within MIRA with the new option /COMPRESS for command SCAN (see MIRA language help).

• For calibration measurements without sky counts, MIRA now issues the receiver temperature(s).

 


Version 1.7

• After extensive testing with the G95 compiler, all memory leaks within MIRA have been removed.

• Under certain weather conditions, TelCal is not very tolerant regarding inaccurate calibration parameters (e.g. forward efficiency, cold load temperature). The new command MIRA\OVERRIDE allows to manually set the load temperatures and beam and forward efficiencies to values different from the entries in the imbFits raw data file. For details, please read the MIRA language internal help.

• The cookbook section now contains a subsection on OTF and HERA data reduction, and another on the reduction of polarisation data.

• Pointing fits now also display the fitted width and peak flux of the Gaussian. Pointing corrections: both the previous and new corrections are displayed.

• Thanks to recent developments in the telcal library (pety@iram.fr), double beam fits for pointings using the wobbler are now supported.

• Baselines for pointing drifts are now fitted simultaneously with the Gaussian. The baseline determined by MIRA (using dumps without the source signal) is used as an initial guess.

• CLASS output files get the default extension .30m

• For pointing drifts, all subscans are now written to the CLASS output file (i.e. option WRITE /SUBSCAN is not needed anymore). See the MIRA language internal help, comand MIRA\WRITE, for details.

• For XPOL observations (VESPA in polarimetry mode), MIRA can calibrate phases. The XPOL widgets for CLASS know whether MIRA has done a phase calibration (CLASS header variable CMODE = 10) or not (CMODE = 0). See the MIRA language internal help, command MIRA\CAL, for details.

• Command MIRA\VIEW gets a new option /XPOL for viewing the amplitude and phase of the cross correlation of orthogonally polarized receivers (for VESPA data in XPOL mode).

• The SIC structure LIST has been simplified: the structure LIST%ATTRIBUTE1 becomes LIST1, LIST%ATTRIBUTE2 becomes LIST2 etc. For details see section 5.2.11.

• There is now one MONITOR section per frontend-backend combination, allowing to accomodate monitor data points that are specific for a given frontend. For details, see section 5.2.6.

• VIEW /CAL now also issues the calibration and weather parameters on the GREG display (so they become visible even if MIRA is used in automatic mode in the background). For HERA data, these parameters are only displayed if option VIEW /CAL /PIXEL is used (otherwise the plots are overcharged).

 


Version 1.8

• Removed a bug: now more than 9 simultaneous frontend-backend combinations allowd.
• Command FIND /NEW now behaves differently: if new data are found, they are appended to the index list (if existing). This allos to load scans that are not in the index list, but whose data files are already written to disk (i.e. avoids a new FIND command in order to get those data).

• New option /SHOW for MIRA command FILE: FILE /SHOW displays the names of the current input and output files (if openend, otherwise a warning message is issued).

• Calibrations of the sideband ratio are now automatically evaluated when MIRA reads in a corresponding scan. The resulting gain image may then be used for further calibration by using the new keyword GAIN[IMAGE] for MIRA command OVERRIDE.

• On-offs with the continuum backend, when displayed with VIEW /PHASE, yield the flux and its error. A baseline linearly depending on elapsed time is automatically fitted by MIRA at the on-off subtraction of MIRA\CAL.

• MIRA now solves skydips.

• Format of backend data becomes dynamic: integer, single or double precision floating point count rates, according to the TFORM3 keyword in the imbFits backend HDU.

• New options /PROJECT and /STAT for command MIRA\FIND, for selecting data from one or several projects, and for writing the total integration time of the observtions of the current index list into a hidden file .stat. Mainly for observing pool administration. LIST /PROJECT includes the project number into the index list.

• Calibrations with less then 3 subscans (i.e. from PAKO\CAL /SKY NO are also plotted, and relevant calibration parameters are indicated on MIRA's graphical display.

• For image gain calibrations of continuum backends, the data are plotted by subscan. For spectrometers, the image gain vs. frequency is shown.

• Command LIST /REDUCE write a procedure for off-line data reduction, instead of listing the data. For details see the MIRA language documentation.

• Option /MASK for command CAL /OFF also uses the OTF dumps as reference (masking the source). Works with OTF maps sufficiently exceeding the emission region.

• New option /TAU for command SCAN. For calibration scans only. Allows to keep the opacity fixed and calculates the corresponding atmospheric model (instead of computing that corresponding to the measured sky emission temperature). Mainly used for skydips (see next item).

• Skydips are now solved by a two-step procedure: First, fit totalPower = a.[1-exp(-tauZenith.airMass)]+b with fit parameters a and tauZenith. Then calibrate the skydip with the atmospheric model corresponding to the measured zenith opacity, and derive the forward efficiency from the fit parameters a and b.

• New option /TRACES for command VIEW for visualizing the antenna speed, fast and slow traces. For identifying bad antenna data.

Version 1.9

• OVERRIDE now allows to set the calibration parameters once for a whole MIRA session, until OVERRIDE /RESET. After reset, a new calibration has to be read.

• VIEW /TRACES now also shows the antenna mount drive dumps between the subscans (the previous versions only showed those data actually used for interpolation to the backend record times). To this end, a new structure has been created that can be examined and visualized via its SIC copy in ANTENNA%ENCODER_AZ_EL (encoder values for azimuth, elevation and their timestamp) and ANTENNA%TRACKING_AZ_EL (tracking errors).

• New parameter calCheck (logical SIC variable) can be used to switch off the consistency check for an observation and its associated calibration measurement (default: .true.). Use let calCheck .false. if you connect more than one part of the same backend to the same receiver, with identical spectral setup. With let calCheck = .true.\, MIRA tries to find for an observation the corresponding spectral backend part in the calibration data, even if its number has changed (e.g. because a backend part was dropped between the calibration and observation).

• In this context, CAL ifb /FEBECAL jfb allows the user to calibrate the frontend-backend unit ifb with the channel gains and calibration temperature of frontend-backend unit jfb (for particular circumstances only, e.g. inconsistent calibrations).

• For pointings, SOLVE ifb jfb computes the alignment of frontend jfb with respect to frontend ifb, and writes the results into an ASCII file alignment.dat (or appends them to it, if existing). The receiver alignment is computed from fits to the difference signal (rather from the differences between fit signals). The results are plotted for a Nasmyth focus cabin.

• Calibrations with switch mode FREQUENCY are supported, but still experimental. Please contact the author for further enquiries.

• New command FLAG allows to flag selected basebands, channels, HERA pixels, records or subscans (e.g. to solve pointings with a bad subscan) .

• New option LIST /FLUX prepares an ASCII table with pointing fluxes in the current index list (calibrated in Ta* if the index list contains calibrations).

• Calibration parameters can be stored in an XML file if writeXML is set: let writeXML yes.

• fix atmospheric water vapour content for calibration: scan /pwv value. Preparations for EMIR nomenclature.

• Observing mode DIY (OTF along Lissajous figures) now allowed. Plot figures with VIEW /OTF.

• Plot limits for tracking errors (command VIEW /TRACES) are variable. This is to suppress visualization of data with too large tracking errors (e.g. while telescope is slewing).

• Skyslews can be read by MIRA and plotted. Fitting opacity and forward efficiency still needs to be developed.

• Doppler correction sign changed on 2008-10-29, to where is the Doppler correction (CLASS variable DOPPLER in the spectroscopy section, and the observatory velocity with respect to the local standard of rest ( if the observator recedes from the LSR).

• Command list /flux argument produces an ASCII output file with continuum fluxes (where argument is the backend name, e.g. 1MHz). list /flux deduces pointing fluxes, using the continuum backends, as before.

• Command cal /off /mask now also works with option /off time. Syntax for /mask d1 [d2 d3 d4 ...] has changed: if only argument d1 is given, this is the number of OTF dumps at the start and end of each OTF subscan used as atmospheric refernce. If (d1,d2), (d3,d4), etc. are given pairwise this is to indicate the respective first and last OTF dump framing the source (up to 5 masks allowed, i.e. 10 arguments).

Version 2.0

• Support for the new EMIR receiver.
• It is now possible to write calibrated total power data to CLASS. Please use:
  cal all /gains /off none /tcal
• It is possible to concatenate spectral subbands for plotting. Command syntax: VIEW ifb1 ifb2 for concatenating subbands ifb1, ifb2, or VIEW ifb1 to ifb2 for concatenating all subbands between ifb1 and ifb2.
• The following parameter can now be set in interactive mode: badLevel, this is the level with respect to median total power at which spectral channels are blanked, default: factor 20 above or below median).
• For continuum observations, the behaviour of view ifb /phase iphase now is the following: iphase = 1, 2,... shows only total power phase iphase. For one of iphase = on, off (for continuum on-offs) or iphase = sky, load (for beam-switched scans) iphase is the leading phase in the signal difference.
• MIRA\override now also allows us to change the weather data (e.g. if the weather stations fails and if wrong weather data are written into the imbfits files). Use: override tamb[_p_humid] Tamb Pamb where Tamb and Pamb are the values for the ambient outside temperature and pressure ( C and hPa, respectively) e.g. measured by hand or * to remain unchanged).
• From 2009-06-17 11:00 UTC onwards, all CLASS spectra written with Mira versions under gildas installations gag_dev and gag_may09 at the 30m telescope are written with the following convention: doppler = -vSource(Obs)/clight, where vSource(Obs) is the source velocity in the observatory frame, in the same sign convention as in Astro (vSource(Obs) < 0 for source approaching observatory).

Version 2.1

• The parameter flagBSwSpikes allows to automatically blank spikes with abnormally short integration times (shorter than actual values, hence normalization wrong, resulting in spikes). Default: .false. (works both for old and EMIR data).
• In the SIC variable section RAW, subsection ANTENNA, the new variables dateObs and dateEnd are the start and end times of a given subscan, as written into the imbfits raw data (i.e., unlike subScanStart and subScanEnd, not filtered by the safeguards in Mira's subroutine syncData).
• If CAL /off NONE was specified, MIRA\WRITE writes both on-source and reference spectra to the output CLASS file.
• New parameter timingCheck (logical flag, default: yes). Allows to switch off the automatic avoidance of hazardous extrapolation of antenna trace data). For details see section 5.1.

Version 2.2

• Full support of 64bit processors. Tested with g95 and ifort compilers on Debian v. 5.0.3.
• Support of data reduction of incomplete imbfits files (i.e., by subscan), except for focus and skydip scans. NB: Currently WILMA and VESPA data are written to imbfits output only at scan end.
• New parameter traceExtrapolation (logical flag, default: yes). Only for timingCheck = no. Enables extrapolation of fast and slow traces before or after scan start, respectively end, and points in time that still contain valid backend data. For details see section 5.1.

Version 2.3

• Data reduction by subscan (pointings, on-off measurements, on-the-fly maps) is now fully supported (for backends whose raw data are written subscanwise).
• New option /BACKEND for MIRA\SCAN select backends from index list for reading and further processing (default: all backends). /DROP drops the selected backends from the list of data to be read and processed.