set_cycle_and_obs_timeinfo module

check_temporal_consistency_cumul_fields(vx_config, cycle_start_times, fcst_len, fcst_output_intvl)

This function reads in a subset of the parameters in the verification configuration dictionary and ensures that certain temporal constraints on these parameters are satisfied. It then returns an updated version of the verification configuration dictionary that satisfies these constraints.

The constraints are on the accumulation intervals associated with the cumulative field groups (and the corresponding observation types) that are to be verified. The constraints on each such accumulation interval are as follows:

The accumulation interval is less than or equal to the forecast length. This ensures that the forecast(s) can accumulate the field(s) in the field group over that interval.
The obs availability interval evenly divides the accumulation interval. This ensures that the obs can be added together to obtain accumulated values of the obs field, e.g. the 6-hourly NOHRSC obs can be added to obtain 24-hour observed snowfall accumulations. Note that this also ensures that the accumulation interval is greater than or equal to the obs availability interval.
The forecast output interval evenly divides the accumulation interval. This ensures that the forecast output can be added together to obtain accumulated values of the fields in the field group. For example, if the forecast output interval is 3 hours, the resulting 3-hourly APCP outputs from the forecast can be added to obtain 6-hourly forecast APCP. Note that this also ensures that the accumulation interval is greater than or equal to the forecast output interval.
The hour-of-day at which the accumulated forecast values will be available are a subset of the ones at which the accumulated obs values are available. This ensures that the accumulated fields from the obs and forecast are valid at the same times and thus can be compared in the verification.

If for a given field-accumulation combination any of these constraints is violated, that accumulation is removed from the list of accumulations to verify for that field.

Parameters:

vx_config (dict) – The verification configuration dictionary.
cycle_start_times (list) – List containing the starting times of the cycles in the experiment; each list element is a datetime object.
fcst_len (datetime.timedelta) – The length of each forecast; a timedelta object.
fcst_output_intvl (datetime.timedelta) – Time interval between forecast output times; a timedelta object.

Returns:

vx_config (dict) – An updated version of the verification configuration dictionary.
fcst_obs_matched_times_all_cycles_cumul (dict) – Dictionary containing the times (in YYYYMMDDHH string format) at which various field/accumlation combinations are output and at which the corresponding obs type is also available.

get_obs_retrieve_times_by_day(vx_config, cycle_start_times, fcst_len, fcst_output_times_all_cycles, obs_days_all_cycles)

This function generates dictionary of dictionaries that, for each combination of obs type needed and each obs day, contains a string list of the times at which that type of observation is needed on that day. The elements of each list are formatted as ‘YYYYMMDDHH’.

Parameters:

vx_config (dict) – The verification configuration dictionary.
cycle_start_times (list) – List containing the starting times of the cycles in the experiment; each list element is a datetime object.
fcst_len (datetime.timedelta) – The length of each forecast.
fcst_output_times_all_cycles (dict) – Dictionary containing a list of forecast output times over all cycles for instantaneous fields and a second analogous list for cumulative fields. Each element of these lists is a string of the form ‘YYYYMMDDHH’.
obs_days_all_cycles (dict) – Dictionary containing a list of observation days (i.e. days on which observations are needed to perform verification) over all cycles for instantaneous fields and a second analogous list for cumulative fields. Each element of these lists is a string of the form ‘YYYYMMDD’.

Returns:

obs_retrieve_times_by_day (dict) – Dictionary of dictionaries containing times at which each type of obs is needed on each obs day.

set_cycle_dates(start_time_first_cycl, start_time_last_cycl, cycl_intvl, return_type='string')

This file defines a function that returns a list containing the starting times of all the cycles in the experiment.

If return_type is set to ‘string’ (the default value), the returned list contains strings in the format ‘YYYYMMDDHH’. If it is set to ‘datetime’, the returned list contains a set of datetime objects.

Parameters:

start_time_first_cycl (datetime.datetime) – Starting time of first cycle.
start_time_last_cycl (datetime.datetime) – Starting time of last cycle.
cycl_intvl (datetime.timedelta) – Time interval between cycle start times.
return_type (str) – Type of the returned list. Can be ‘string’ or ‘datetime’.

Returns:

all_cdates (list) – Either a list of strings in the format ‘YYYYMMDDHH’ or a list of datetime objects containing the cycle starting times, where ‘YYYY’ is the four- digit year, ‘MM is the two-digit month, ‘DD’ is the two-digit day-of- month, and ‘HH’ is the two-digit hour-of-day.

set_fcst_output_times_and_obs_days_all_cycles(cycle_start_times, fcst_len, fcst_output_intvl)

This function returns forecast output times and observation days (i.e. days on which obs are needed because there is forecast output on those days) for both instantaneous (e.g. REFC, RETOP, T2m) and cumulative (e.g. APCP) fields that need to be verified. Note that for cumulative fields, the only accumulation interval considered is the forecast output interval. Accumulation intervals larger than this are considered elsewhere (and accumulation interval smaller than this are obviously not allowed).

Parameters:

cycle_start_times (list) – List containing the starting times of the cycles in the experiment; each list element is a datetime object.
fcst_len (datetime.timedelta) – The length of each forecast.
fcst_output_intvl (datetime.timedelta) – Time interval between forecast output times.

Returns:

fcst_output_times_all_cycles (dict) – Dictionary containing a list of forecast output times over all cycles for instantaneous fields and a second analogous list for cumulative fields. Each element of these lists is a string of the form ‘YYYYMMDDHH’.
obs_days_all_cycles (dict) – Dictionary containing a list of observation days (i.e. days on which observations are needed to perform verification) over all cycles for instantaneous fields and a second analogous list for cumulative fields. Each element of these lists is a string of the form ‘YYYYMMDD’.

set_rocoto_cycledefs_for_obs_days(obs_days_all_cycles)

Given a list of days on which observations are needed (because there is forecast output on those days), this function generates a list of ROCOTO- style cycledef strings that together span the days (over all cycles of an SRW App experiment) on which obs are needed. The input list of days must be increasing in time, but the days do not have to be consecutive, i.e. there may be gaps between days that are greater than one day.

Each cycledef string in the output list represents a set of consecutive days in the input string (when used inside a <cycledef> tag in a ROCOTO XML). Thus, when the cycledef strings in the output string are all assigned to the same cycledef group in a ROCOTO XML, that group will represent all the days on which observations are needed. This allows the ROCOTO workflow to define a single set of non-consecutive days on which obs are needed and define tasks (e.g. get_obs) only for those days, thereby avoiding the redundant creation of these tasks for any in-between days on which obs are not needed.

Parameters:

obs_days_all_cycles (list) – A list of strings of the form ‘YYYYMMDD’, with each string representing a day on which observations are needed. Note that the list must be sorted, i.e. the days must be increasing in time, but there may be gaps between days.

Returns:

cycledefs_all_obs_days (list) – A list of strings, with each string being a ROCOTO-style cycledef of the form

‘{yyyymmdd_start}0000 {yyyymmdd_end}0000 24:00:00’

where {yyyymmdd_start} is the starting day of the first cycle in the cycledef and {yyyymmdd_end} is the starting day of the last cycle (note that the minutes and hours in these cycledef stirngs are always set to ‘00’). For example, an element of the output list may be:

‘202404290000 202405010000 24:00:00’