Astronomical exposure reduction (and) image stacking modus

User Manual

Copyright 2017 -18

Asterism is a Linux (operating system) astronomical image preprocessing utility, designed with consumer digital camera (CDC) RAW data and many other image formats in mind. Asterism makes use of freely available image manipulation programs, common to Linux distributions.

Asterism is free and was developed to see whether an authentic image calibration work-flow could be achieved with existing image processing programs - DCRAW ImageMagick and Panotools.

Most of Asterism’s processes are independent or stand-alone and may be used in virtually any combination. However, best results are achieved by following the suggestions outlined in this manual.

Noise reduction methods have been tested on a variety of image sets and the best methods (found thus far) incorporated into Asterism’s work-flow. However, the user retains control of Asterism’s processes, as much as practicable.

Note: Newcomers may benefit from an explanation of the terms used in this document – please refer to the next section of this manual.

CDC image processing benefits from a flexible approach. Image sets are often unique and of variable quality. The ability to adapt and choose from a variety of processes is advantageous. Experimenting and comparing results is inevitable.

The most effective method for calibrating cooled RAW data (acquired at and below 0°C – though +5°C is not out of the question), is to use Bias frames and Flat frames. Otherwise, for RAW data (that is not cooled) a Defect map and Flat frames are preferred. The use of Dark frames is optional but a Defect map is a better choice.

Note: The Defect map is created from Dark frames; and because the “bias is in the dark”, a Defect map is effective even when applied to cooled data.

Asterism avoids data depleting processes and should never truncate pixel values. CDC data can be inconsistent and is generally not linear. In-fact, CDC data linearity varies between image types – bias dark, flats and lights. Asterism processes are intended to be safe and predictable.

As CDCs are (usually) one shot colour (OSC) devices – that is, the image sensor is overlaid with a colour mosaic (Bayer matrix) or a colour (Fovean) filter of some sort - the terms, CDC and OSC are interchangeable, for the purpose of this manual.

Installation permissions and system requirements are described in the last sections of this manual.

Terms in use

Several terms are interchangeable in astrophotography speak - sometimes in error. The newcomer will get used to this with experience. Many will recognise image preprocessing as signal processing; and for the purpose of this manual.

RAW – RAW data or image data. The image file or data file straight from the camera in its RAW format (a matrix of numbers defining Red Green and Blue values – that is, no processing has been performed on the image.

Note: For the purpose of this manual, we consider RAW data to be (nearly) linear; that is, the image has not been developed - analogous to film, which requires a chemical process to reveal the picture taken by the photographer - RAW data must be processed to show the image detail.

Noise – unwanted signal present in images produced by an electronic image sensor.

Noise reduction – various methods of filtering or reducing unwanted signal in astronomical images.

Light frames – the images of the astronomical object – usually, multiple images, combined to obtain a higher signal to noise ratio (SNR).

Bias frames – images of the electronic ‘stamp’ of the camera image sensor in operation – fixed pattern and predictable.

Dark frames – images of the dark current / noise generated while the sensor is operating – fixed pattern and predictable at a given temperature, exposure time and ISO setting.

Flat frames – images of the optical flaws (noise) obstructions, dust, hair and other contaminants, between the telescope objective lens and the image sensor. Take these after every image session.

Master frames - Templates or Master frames, representing the bias dark and flat signal, are created separately by combining the bias dark and flat frames – and are referred to, as Master bias Master dark and Master flat.

Defect map – A template used to reduce / remove the effect of very bright and very dark pixels – these are usually artefacts and detract from the appearance of the final image. For example, hot and cold pixels.

Preprocessing – image calibration (also known as image reduction); that is, preparation of Master frames and removal of unwanted signal (noise) from Light frames, followed by deBayering, image alignment and stacking (registration or combination, as it is also known), of the Light frames.

Using Asterism – overview

Using Asterism is as simple as extracting the downloaded zip file to the user’s home directory (~/Asterism), opening a terminal in the Asterism folder and typing ./asterism or bash -x ./asterism to view the entire bash process.

A very basic RAW preprocessing work flow; as follows

1. Choose or create a Project folder* – usually the folder containing the image files - but the image files may be located anywhere.

Note: the project name is automatically assigned the name of the project folder – which must be all one word – no spaces

2. Check ‘Input format’ set to RAW (default).

3. Select the required processes and options accordingly

4. Accept all other default settings; and

5. Select ‘Process’ on the bottom tool bar.

This is a minimalist approach. The essential switches have been selected and Asterism will perform the selected task/s and return to the main interface.

Important note: The mean-shift noise reduction method is applied to all image calibration processes. If using the mean-shift method, create the Master frames first and then process the Light frames. The ‘radius’ and / or ‘amount’, used for noise reduction in Master frames, may be too aggressive for Light frames.

Note: By default, all calibration frames (images) are mean (average) combined (to create Master frames). However, median combination produces better noise rejection in the Master frames, for less noise contamination of Light frames, during image calibration; alternatively, use the mean-shift method as mentioned above.

A Super-bias is preferred for calibration of Flat frames and Light frames.

A noise reduction work flow may involve creating a Defect map (or Super-bias) and Master flat and, if need be, the application of mean-shift noise reduction and / or setting minimum / maximum pixel values with QRMin and QRMax to the light frames.

Note: QR = quantum range; that is, 0 - 65535 or 65536 colour values for a 16 bit image.

QRMin and QRMax reassign minimum and maximum pixel values to pixels outside the percentage range selected.

Important note: Do not use QRMin QRMax when processing master frames and light frames in the same processing run. Create the master frames separately from the light frames.

Noise rejection in the light frames is further enhanced by polynomial stacking, to produce a clean image free of overly light and dark pixels / speckles / salt and pepper.

Note: Dark frames will not be used if a ‘Defect map’ is loaded.

A small image set of 3 bias 3 darks (optional), 3 flats and 3 lights (or as many as considered representative) is a convenient way to gauge the likely outcome of the full image set. Particularly the noise reduction characteristics of the chosen process.

Asterism’s processing model

This section is provided for completion and is helpful understanding CDC / OSC RAW image (data) reduction.

A goal of preprocessing is to produce a stacked image with a high signal to noise ratio (SNR).

As previously mentioned, the Asterism preprocessing model assumes that CDC RAW data is not strictly linear (intended to be data safe).

A work flow suitable for cooled data is;

Signal = (Light Frames – Master Bias) / (Flat Frames – Master Bias)

Traditionally, selecting ‘Dark Frames’ or ‘Master dark’ modifies the algorithm for cameras without cooling;

Signal = (Light Frames – Master Dark) / (Flat Frames – Master Bias)

Note: CDC Dark frames can be problematic and some astrophotographers prefer not to use them. Similarly, modern image sensors tend to exhibit less dark current; therefore, Dark frames remain optional.

Note: Flat frames have a significant bearing on image quality, apart from the removal of optical artefacts. Good flats make all the difference to an image set.

Note: Bias frames are essential to Master flat creation.

Image and calibration frame acquisition

This section is provided for general reference and is not intended to be a complete discussion about image acquisition or calibration.

Light frame acquisition is subject to a range of variables and often, quality is a matter of experience and personal preference. Calibration frames are more predictable.

Bias frames should be taken at the camera’s lowest ISO setting and the fastest shutter speed, with the sensor in complete darkness.

Flat frames should also be taken at the camera’s lowest ISO setting and the same aperture as the light frames. Flat illumination is a function of shutter speed, brightness of the light source and aperture.

As a general guide, flat illumination should be approximately 50 – 70% (right side) of camera live view histogram. 50% for dark skies and 70% for suburban skies, when using Asterism.

Note: Asterism will compensate for under and over exposed flats, as well as variations in light frame exposure.

Darks should be taken with the sensor in complete darkness, at the same ISO, exposure time and temperature as light frames. Regulated cooling provides an advantage and darks may be omitted at and below 0C. There are various strategies to match dark and light frame temperature during acquisition, where cooling is not available.

As previously mentioned, it is preferable to create a super-bias or Defect map and Master flat. Use the noise radius and amount settings for flat creation.

The latest version of Asterism has a Develop function and produces a full resolution background extracted auto white balanced image, as well as the linear unprocessed image.

Develop also presents a 25% preview of the final image, which may be processed and saved using ImageMagick’s display menu. Note: The full resolution images are not changed. The reliability of Develop depends on the quality of input images and calibration method – it’s a preview.

The new interface

Except the ‘Process and options’ tab, all other tabs are unchanged. The new layout is easy to use. Select individual processes with options and press ‘Process’ (bottom toolbar). Other stand-alone functions may also be accessed from the bottom toolbar.

The old version manual has been retained separately. Many of Asterism’s functions have not changed. However, 3 channel RGB processing has been removed from Asterism onwards – use the deprecated version available on the homepage, if required.

Files’ manager - ‘Convert’ image format - ‘Inspect’ blink utility - ‘PMStack‘, Hugin and Histogram are selected from the bottom toolbar.

Note: if starting with ‘Light frames’, Master frames should be selected. If not, a warning appears with the option to ‘Return’ to the Asterism interface or to ‘Continue’ processing without image calibration.

Please refer to the bottom toolbar section for a description of the button functions

Tabs and options

Asterism tabs

Process and options’ - the main tab from which all image processing originates.

Select camera matrix’ - apply ‘sRGB’ channel multipliers – unmodified cameras.

Add camera matrix’ - add up to 5 unique channel multiplier matrices.

Edit camera list’ - remove or modify channel multipliers.

'Project details' - record the details of the project.

'System settings' - remaining hard-drive space, memory and processing threads.

'Documentation' – links to documentation resources and version information.

Process and options tab

*’ is a reminder that an entry is required in the field – mandatory...

Project folder*’ - select or create a project folder – the project name is assigned automatically.

Input format’ - RAW, OTHER (e.g., jpeg png tiff - deBayered images), OTHER BAYER (images that are not RAW and have not been deBayered – e.g., fits tiff).

Output format’ – enter an output extension – default tif (16bit no compression)

Colour depth’ - default 16 – range 8 – 64 in 8 bit steps

Process options

Note: The indented options are selected with the main option; for example, to create a ‘Super-bias’, select ‘Bias frames’ and Super-bias’. Similarly, for median combined frames.

Map Defects’ – Create a ‘Defect map’ -Defect map’ is created from Dark frames – as many as available (1 dark frame is OK). A single dark frame ‘Defect map’ is superior to a single dark. Use Defect map in place of a Master dark.

Bias frames- Create a mean combined Master bias

- ‘Super-bias’ – Create a (master) ‘Super-bias’ - significant random noise reduction in the Master bias frame, results in cleaner calibration of flat and light frames.

- Median’ - Create a median filtered Master bias - ‘Super-bias’ is preferred.

Dark frames’ - Create a mean combined Master dark.

- ‘Median’ - Create a median filtered Master dark, useful for minimising noise in noisy image sets and / or small calibration frame image sets - ‘Defect map’ is preferred.

Flat frames’ - Create a mean combined Master flat. Setting a ‘Noise - radius’ = 7 and ‘Noise - amount’ = 0.1, creates a Master flat with similar standard deviation to a median combined Master flat – the process is much quicker than ‘Median’.

- ‘Median’ - Create a median filtered Master flat, useful for minimising noise in noisy image sets and / or small calibration frame image sets – see above alternative application of ‘Noise reduction parameters’.

Flat correction- Automatic flat correctiondefault ‘5’ for automatic flat correction of all light frames, regardless of variations in flat or light exposure. This process rescues poor flats and mixed light image sets with differing exposures. The luminance of each light image is calculated and the flat frame adjusted to ensure reliable flat reduction. Set ‘0’ to turn off on a session-by-session basis.

Note: If a Master bias is not available to the Flat frames a file selection window will pop-up – select a Master bias – Super-bias preferred.

Note: Flats are assumed to be taken at each imaging session.

Note: Select the .miff extension master frames created in Asterism - found in the master folder located in the project folder), for best results.

Defect map