Processing

main	Main function that initiates and controls the processing workflow.

s1ard.processor.main(config_file=None, debug=False, **kwargs)

Main function that initiates and controls the processing workflow.

Parameters:

• config_file (str or None) – Full path to a config.ini file or None to use the package’s default file.

• debug (bool) – Set logging level to DEBUG? Default is False.

• **kwargs – extra arguments to override parameters in the config file. E.g. acq_mode.

SAR

s1ard offers a mechanism to plug in different SAR processors. The software offers a snap reference implementation module that can be translated to other processors. All “main interface” functions need to be implemented so that s1ard can fully interact with the module.

SNAP

main interface

config_to_string	Convert the values of a configuration dictionary to strings.
get_config_keys	Get all allowed configuration keys.
get_config_section	Get the`config.ini` SNAP section content as a dictionary.
get_metadata	Get processing metadata needed for ARD product metadata.
process	Main function for SAR processing with SNAP.
translate_annotation	Translate s1ard annotation keys to SAR processor naming.

processor-specific functions

grd_buffer	GRD extent buffering.
look_direction	Compute the per-pixel range look direction angle.
nrt_slice_num	Check whether a product has a non-zero slice number and add it if not.
pre	General SAR preprocessing.

s1ard.snap.config_to_string(config)

Convert the values of a configuration dictionary to strings.

Parameters:

config (dict) – the configuration as returned by get_config_section()

Returns:

the dictionary with the same structure but values converted to strings.

Return type:

dict

s1ard.snap.get_config_keys()

Get all allowed configuration keys.

Return type:

List[str]

s1ard.snap.get_config_section(parser, **kwargs)

Get the`config.ini` SNAP section content as a dictionary.

Parameters:

• parser (configparser.ConfigParser)

• kwargs (dict[str])

Return type:

dict

s1ard.snap.get_metadata(scene, outdir)

Get processing metadata needed for ARD product metadata.

Parameters:

• scene (str) – the name of the SAR scene

• outdir (str) – the directory to search for processing output

Return type:

dict

s1ard.snap.grd_buffer(src, dst, workflow, neighbors, buffer=100, gpt_args=None)

GRD extent buffering. GRDs, unlike SLCs, do not overlap in azimuth. With this function, a GRD can be buffered using the neighboring acquisitions. First, all images are mosaicked using the SliceAssembly operator and then subsetted to the extent of the main scene including a buffer. The SliceAssembly operator needs info about the slice number (i.e., the ID/position inside the data take). If the value in the metadata is 0 (which can be the case in NRT slicing mode), the slice number is determined using function nrt_slice_num(). If this fails, the function will raise an error.

Parameters:

• src (str) – the file name of the source scene in BEAM-DIMAP format.

• dst (str) – the file name of the target scene. Format is BEAM-DIMAP.

• workflow (str) – the output SNAP XML workflow filename.

• neighbors (list[str]) – the file names of neighboring scenes

• buffer (int) – the buffer size in meters

• gpt_args (list[str] or None) –

  a list of additional arguments to be passed to the gpt call

  • e.g. ['-x', '-c', '2048M'] for increased tile cache size and intermediate clearing

Raises:

RuntimeError – if the slice number of a scene is 0, and it could not be determined from the acquisition time

s1ard.snap.look_direction(dim)

Compute the per-pixel range look direction angle. This adds a new layer to an existing BEAM-DIMAP product.

Steps performed:

• read geolocation grid points

• limit grid point list to those relevant to the image

• for each point, compute the range direction angle to the next point in range direction.

• interpolate the grid to the full image dimensions

Notes

• The interpolation depends on the location of the grid points relative to the image. Hence, by subsetting the image by an amount of pixels/lines different to the grid point sampling rate, the first and last points will no longer be in the first and last line respectively.

• The list might get very large when merging the scene with neighboring acquisitions using SliceAssembly and this longer list significantly changes the interpolation result. The difference in interpolation can be mitigated by reducing the list of points to those inside the image and those just outside of it.

Parameters:

dim (str) – a BEAM-DIMAP metadata file (extension .dim)

s1ard.snap.nrt_slice_num(dim)

Check whether a product has a non-zero slice number and add it if not. In NRT Slicing mode, both sliceNumber and totalSlices are 0 in the manifest.safe file. sliceNumber is however needed in function grd_buffer() for the SNAP operator SliceAssembly. The time from segmentStartTime to last_line_time is divided by the acquisition duration (last_line_time - first_line_time). totalSlices is set to 100, which is expected to exceed the maximum possible value.

Parameters:

dim (str) – the scene in BEAM-DIMAP format

Raises:

RuntimeError – if the slice number is 0, and it cannot be computed because the segment start time cannot be read from the metadata

s1ard.snap.pre(src, dst, workflow, allow_res_osv=True, osv_continue_on_fail=False, output_noise=True, output_beta0=True, output_sigma0=True, output_gamma0=False, add_slice_num=True, gpt_args=None)

General SAR preprocessing. The following operators are used (optional steps in brackets): Apply-Orbit-File(->Remove-GRD-Border-Noise)->Calibration->ThermalNoiseRemoval(->TOPSAR-Deburst)

Parameters:

• src (str) – the file name of the source scene

• dst (str) – the file name of the target scene. Format is BEAM-DIMAP.

• workflow (str) – the output SNAP XML workflow filename.

• allow_res_osv (bool) – Also allow the less accurate RES orbit files to be used?

• osv_continue_on_fail (bool) – Continue processing if no OSV file can be downloaded or raise an error?

• output_noise (bool) – output the noise power images?

• output_beta0 (bool) – output beta nought backscatter needed for RTC?

• output_sigma0 (bool) – output sigma nought backscatter needed for NESZ?

• output_gamma0 (bool) – output gamma nought backscatter needed?

• add_slice_num (bool) – determine a slice number and add it to the product’s metadata? This is only necessary if GRD buffering is intended. See nrt_slice_num().

• gpt_args (list[str] or None) –

  a list of additional arguments to be passed to the gpt call

  • e.g. ['-x', '-c', '2048M'] for increased tile cache size and intermediate clearing

See also

pyroSAR.snap.auxil.orb_parametrize

s1ard.snap.process(scene, outdir, measurement, spacing, dem, dem_resampling_method='BILINEAR_INTERPOLATION', img_resampling_method='BILINEAR_INTERPOLATION', rlks=None, azlks=None, tmpdir=None, export_extra=None, allow_res_osv=True, clean_edges=True, clean_edges_pixels=4, neighbors=None, gpt_args=None, cleanup=True)

Main function for SAR processing with SNAP.

Parameters:

• scene (str) – The SAR scene file name.

• outdir (str) – The output directory for storing the final results.

• measurement ({'sigma', 'gamma'}) –

  the backscatter measurement convention:

  • gamma: RTC gamma nought (\(\gamma^0_T\))

  • sigma: RTC sigma nought (\(\sigma^0_T\))

• spacing (int or float) – The output pixel spacing in meters.

• dem (str) – The DEM filename. Can be created with s1ard.dem.mosaic().

• dem_resampling_method (str) – The DEM resampling method.

• img_resampling_method (str) – The image resampling method.

• rlks (int or None) – The number of range looks.

• azlks (int or None) – The number of azimuth looks.

• tmpdir (str or None) – Path to a temporary directory for intermediate products.

• export_extra (list[str] or None) –

  A list of ancillary layers to create. Default None: do not create any ancillary layers. Options:

  • DEM

  • gammaSigmaRatio: \(\sigma^0_T / \gamma^0_T\)

  • sigmaGammaRatio: \(\gamma^0_T / \sigma^0_T\)

  • incidenceAngleFromEllipsoid

  • layoverShadowMask

  • localIncidenceAngle

  • NESZ: noise equivalent sigma zero

  • projectedLocalIncidenceAngle

  • scatteringArea

  • lookDirection: range look direction angle

• allow_res_osv (bool) – Also allow the less accurate RES orbit files to be used?

• clean_edges (bool) – Erode noisy image edges? See pyroSAR.snap.auxil.erode_edges(). Does not apply to layover-shadow mask.

• clean_edges_pixels (int) – The number of pixels to erode.

• neighbors (list[str] or None) – (only applies to GRD) an optional list of neighboring scenes to add a buffer around the main scene using function grd_buffer(). If GRDs are processed compeletely independently, gaps are introduced due to a missing overlap. If neighbors is None or an empty list, buffering is skipped.

• gpt_args (list[str] or None) –

  a list of additional arguments to be passed to the gpt call

  • e.g. ['-x', '-c', '2048M'] for increased tile cache size and intermediate clearing

• cleanup (bool) – Delete intermediate files after successful process termination?

Examples

>>> from s1ard import snap
>>> scene = 'S1A_IW_SLC__1SDV_20200103T170700_20200103T170727_030639_0382D5_6A12.zip'
>>> dem = 'S1A_IW_SLC__1SDV_20200103T170700_20200103T170727_030639_0382D5_6A12_DEM_EEA10.tif'
>>> outdir = '.'
>>> spacing = 10
>>> rlks = 5
>>> azlks = 1
>>> export_extra = ['localIncidenceAngle', 'incidenceAngleFromEllipsoid',
>>>                 'scatteringArea', 'layoverShadowMask', 'gammaSigmaRatio']
>>> snap.process(scene=scene, outdir=outdir, spacing=spacing, dem=dem,
>>>              rlks=rlks, azlks=azlks, export_extra=export_extra)

s1ard.snap.translate_annotation(annotation, measurement)

Translate s1ard annotation keys to SAR processor naming.

Parameters:

• annotation (List[str]) – the s1ard annotation keys (e.g. ei, gs)

• measurement (str) – the SAR backscatter measurement (gamma|sigma)

Returns:

the annotation layer keys as required by the SAR processor

Return type:

List[str]

ARD

calc_product_start_stop	Calculates the start and stop times of the ARD product.
create_acq_id_image	Creation of the Acquisition ID image.
create_data_mask	Creation of the Data Mask image.
create_rgb_vrt	Creation of the color composite GDAL VRT file.
create_vrt	Create a GDAL VRT file executing an on-the-fly pixel function.
format	Create ARD products from the SAR processor output.
get_datasets	Collect processing output for a list of scenes.
wind_normalization	Create ORB wind normalization layers.

s1ard.ard.append_metadata(config, prod_meta, src_ids, assets, compression, wm_ref_speed=None, wm_ref_direction=None)

Append metadata files to an ARD product.

Parameters:

• config (dict[str, dict[str, int | float | str | list[str]]]) – the configuration dictionary

• prod_meta (dict[str, str | int | datetime]) – the product metadata as returned by product_info()

• src_ids (list[ID]) – the source product objects

• assets (list[str]) – a list of assets in the ARD product as returned by format().

• compression (str) – the used compression algorithm

• wm_ref_speed (list[str] | None) – the wind model reference wind speed files

• wm_ref_direction (list[str] | None) – the wind model reference wind direction files

Return type:

None

s1ard.ard.format(config, prod_meta, src_ids, sar_assets, tile, extent, epsg, wbm=None, dem_type=None, multithread=True, compress=None, overviews=None, annotation=None)

Create ARD products from the SAR processor output. This includes the following:

• Creating all measurement and annotation datasets in Cloud Optimized GeoTIFF (COG) format

• Creating additional annotation datasets in Virtual Raster Tile (VRT) format

• Applying the ARD product directory structure & naming convention

• Generating metadata in XML and JSON formats for the ARD product as well as source SLC datasets

Parameters:

• config (dict[str, dict[str, int | float | str | list[str]]]) – Dictionary of the parsed config parameters for the current process.

• prod_meta (dict[str, str | int | datetime]) – Product metadata as returned by product_info().

• src_ids (list[ID]) – List of scenes to process. Either a single scene or multiple, matching scenes (consecutive acquisitions). All scenes are expected to overlap with extent and an error will be thrown if the processing output cannot be found for any of the scenes.

• sar_assets (list[dict[str, str]]) – The SAR processing assets as returned by get_datasets().

• tile (str) – ID of an MGRS tile.

• extent (dict[str, int | float]) – Spatial extent of the MGRS tile, derived from a Vector object.

• epsg (int) – The CRS used for the ARD product; provided as an EPSG code.

• wbm (str | None) – Path to a water body mask file with the dimensions of an MGRS tile.

• dem_type (str | None) – if defined, a DEM layer will be added to the product. The suffix em (elevation model) is used. Default None: do not add a DEM layer.

• multithread (bool) – Should gdalwarp use multithreading? Default is True. The number of threads used, can be adjusted in the config.ini file with the parameter gdal_threads.

• compress (str | None) – Compression algorithm to use. See https://gdal.org/drivers/raster/gtiff.html#creation-options for options. Defaults to ‘LERC_DEFLATE’.

• overviews (list[int] | None) – Internal overview levels to be created for each GeoTIFF file. Defaults to [2, 4, 9, 18, 36]

• annotation (list[str] | None) –

  an optional list to select the annotation layers. Default None: create all layers if the source products contain the required input layers. Options:

  • dm: data mask (four masks: not layover not shadow, layover, shadow, water)

  • ei: ellipsoidal incident angle

  • em: digital elevation model

  • id: acquisition ID image (source scene ID per pixel)

  • lc: RTC local contributing area

  • ld: range look direction angle

  • li: local incident angle

  • np: noise power (NESZ, per polarization)

  • gs: gamma-sigma ratio: sigma0 RTC / gamma0 RTC

  • sg: sigma-gamma ratio: gamma0 RTC / sigma0 ellipsoidal

  • wm: OCN product wind model; requires OCN scenes via argument scenes_ocn

Return type:

list[str]

Returns:

the ARD product assets

s1ard.ard.get_datasets(scenes, sar_dir, extent, epsg, processor_name)

Collect processing output for a list of scenes. Reads metadata from all source SLC/GRD scenes, finds matching output files in sar_dir and filters both lists depending on the actual overlap of each SLC/GRD valid data coverage with the current MGRS tile geometry. If no output is found for any scene the function will raise an error. To obtain the extent of valid data coverage, first a binary mask raster file is created with the name datamask.tif, which is stored in the same folder as the processing output as found by find_datasets(). Then, the boundary of this binary mask is computed and stored as datamask.gpkg (see function spatialist.vector.boundary()). If the provided extent does not overlap with this boundary, the output is discarded. This scenario might occur when the scene’s geometry read from its metadata overlaps with the tile but the actual extent of data does not.

Parameters:

• scenes (list[str]) – List of scenes to process. Either an individual scene or multiple, matching scenes (consecutive acquisitions).

• sar_dir (str) – The directory containing the SAR datasets processed from the source scenes using pyroSAR. The function will raise an error if the processing output cannot be found for all scenes in sar_dir.

• extent (dict[str, int | float]) – Spatial extent of the MGRS tile, derived from a Vector object.

• epsg (int) – The coordinate reference system as an EPSG code.

• processor_name (str) – The name of the used SAR processor. The function find_datasets of the respective processor module is used.

Return type:

tuple[list[ID], list[dict[str, str]]]

Returns:

List of ID objects of all source SLC/GRD scenes that overlap with the current MGRS tile and a list of SAR processing output files that match each ID object of ids. The format of the latter is a list of dictionaries per scene with keys as described by e.g. s1ard.snap.find_datasets().

See also

s1ard.snap.find_datasets()

s1ard.ard.product_info(product_type, src_ids, tile_id, extent, epsg, dir_ard, update=False, product_id=None)

Create initial ARD product metadata and create the product’s target directory.

Parameters:

• product_type (str) – the ARD product type; options: NRB, ORB

• src_ids (list[ID]) – the source product objects

• tile_id (str) – the MGRS tile ID

• extent (dict[str, int | float]) – the extent of the MGRS tile

• epsg (int) – the EPSG code of the MGRS tile

• dir_ard (str) – the ARD output directory

• update (bool) – update an existing product (or create a new one)

• product_id (str | None) – an existing product ID. Default None: create a new one using function generate_unique_id().

Return type:

dict[str, str | int | datetime] | None

Returns:

ARD product metadata or None if the determined product’s time range is outside the valid acquisition time range or the product already exists.

s1ard.ard.wind_normalization(src, dst_wm, dst_wn, measurement, gapfill, bounds, epsg, driver, creation_opt, dst_nodata, multithread, resolution=915)

Create ORB wind normalization layers. A wind model annotation layer is created and optionally a wind normalization VRT.

Parameters:

• src (list[str]) – A list of OCN products as prepared by s1ard.ocn.extract()

• dst_wm (str) – The name of the wind model layer in the ARD product

• dst_wn (str | None) – The name of the wind normalization VRT. If None, no VRT will be created. Requires measurement to point to a file.

• measurement (str | None) – The name of the measurement file used for wind normalization in dst_wn. If None, no wind normalization VRT will be created.

• gapfill (bool) – Perform additional gap filling (s1ard.ocn.gapfill())? This is recommended if the Level-1 source product of measurement is GRD in which case gaps are introduced between subsequently acquired scenes.

• bounds (list[float]) – the bounds of the MGRS tile

• epsg (int) – The EPSG code of the MGRS tile

• driver (str) – GDAL driver to use for raster file creation.

• creation_opt (list[str]) – GDAL creation options to use for raster file creation. Should match specified GDAL driver.

• dst_nodata (float) – Nodata value to write to the output raster.

• multithread (bool) – Should gdalwarp use multithreading?

• resolution (int) – The target pixel resolution in meters. 915 is chosen as default because it is closest to the OCN product resolution (1000) and still fits into the MGRS bounds (109800 % 915 == 0).

Return type:

None

ETAD

process

Apply ETAD correction to a Sentinel-1 SLC product.

s1ard.etad.process(scene, etad_dir, out_dir)

Apply ETAD correction to a Sentinel-1 SLC product.

Parameters:

• scene (pyroSAR.drivers.ID) – The Sentinel-1 SLC scene.

• etad_dir (str) – The directory containing ETAD products. This will be searched for products matching the defined SLC.

• out_dir (str) – The directory to store results. The ETAD product is unpacked to this directory if necessary. Two new sub-directories SLC_original SLC_ETAD and are created, which contain the original unpacked scene and the corrected one respectively.

Returns:

The corrected scene as a pyroSAR ID object.

Return type:

pyroSAR.drivers.ID

OCN

extract	Extract an OCN product's image variable and write it to a new GeoTIFF file.
gapfill	Fill gaps of an image file using GDAL.

s1ard.ocn.extract(src, dst, variable)

Extract an OCN product’s image variable and write it to a new GeoTIFF file. Coordinates are extracted from the corresponding latitude and longitude image variables and the corner coordinates written as ground control points (GCPs) to the output file.

Parameters:

• src (str) – path to OCN product SAFE folder

• dst (str) – the name of the GeoTIFF file to write

• variable (str) – name of the layer to extract from the OCN product, e.g. owiNrcsCmod

s1ard.ocn.gapfill(src, dst, md, si)

Fill gaps of an image file using GDAL.

Parameters:

• src (str) – the source image file

• dst (str) – the destination image file with gaps filled

• md (int) – the interpolation maximum distance

• si (int) – the number of smoothing iterations

See also

osgeo.gdal.FillNodata