.. _AQM: ===================================== Air Quality Modeling (UFS-AQM) ===================================== .. attention:: AQM capabilities are an unsupported feature of the SRW App. This means that it is available for users to experiment with, but assistance for AQM-related issues is limited. The standard SRW App distribution uses the uncoupled version of the UFS Weather Model (atmosphere-only). However, users have the option to use a coupled version of the SRW App that includes the standard distribution (atmospheric model) plus the Air Quality Model (AQM). The AQM is a UFS Application that dynamically couples the Community Multiscale Air Quality (:term:`CMAQ`) model with the UFS Weather Model (WM) through the :term:`NUOPC` Layer to simulate temporal and spatial variations of atmospheric compositions (e.g., ozone and aerosol compositions). The CMAQ model, treated as a column chemistry model, updates concentrations of chemical species (e.g., ozone and aerosol compositions) at each integration time step. The transport terms (e.g., :term:`advection` and diffusion) of all chemical species are handled by the UFS WM as tracers. .. note:: Although this chapter is the primary documentation resource for running the AQM configuration, users may need to refer to :numref:`Chapter %s ` and :numref:`Chapter %s ` for additional information on building and running the SRW App, respectively. Quick Start Guide (UFS-AQM) ===================================== .. attention:: These instructions should work smoothly on Ursa, Hercules, Orion, Derecho, and Gaea-C6 but users on other systems may need to make additional adjustments. Download the Code ------------------- Clone the |branch| branch of the authoritative SRW App repository: .. include:: ../../doc-snippets/clone.rst Checkout Externals --------------------- Users must run the ``checkout_externals`` script to collect (or "check out") the individual components of the SRW App (AQM version) from their respective GitHub repositories. .. include:: ../../doc-snippets/externals.rst Build the SRW App with AQM ----------------------------- Users can build the SRW App AQM binaries with the following command: .. code-block:: console ./devbuild.sh -p= -a=ATMAQ where ```` is ``ursa``, ``hercules``, ``orion``, ``derecho``, and ``gaeac6``. The ``-a`` argument indicates the configuration/version of the application to build. Building the SRW App with AQM on other machines, including other :srw-wiki:`Level 1 ` platforms, is not currently guaranteed to work, and users may have to make adjustments to the modulefiles for their system. If the SRW-AQM builds correctly, users should see the standard executables listed in :numref:`Table %s `. Additionally, users should see the AQM-specific executables described in :numref:`Table %s ` in the ``ufs-srweather-app/exec`` directory. .. _AQM-exec: .. list-table:: *Names and descriptions of additional executables produced when the ATMAQ option is enabled* :widths: 20 50 :header-rows: 1 * - Executable - Description * - decomp-ptemis-mpi - Splits the point-source emission file into subdomain based on runtime configure setting * - gefs2lbc_para - Interpolates :term:`GOCART` concentration to be lateral boundary condition for regional air quality model and outputs a layer result for checking purpose * - nexus - Runs the NOAA Emission and eXchange Unified System (:ref:`NEXUS `) emissions processing system Load the |wflow_env| Environment -------------------------------------------- Load the python environment for the workflow: .. code-block:: console # On all systems: module use /path/to/ufs-srweather-app/modulefiles module load wflow_ where ```` is ``ursa``, ``hercules``, ``orion``, ``derecho``, or ``gaeac6``. The workflow should load on other platforms listed under the ``MACHINE`` variable in :numref:`Section %s `, but users may need to adjust other elements of the process when running on those platforms. .. note:: A limited number of diagnostic fields are output on ``derecho`` due to runtime CMAQ issues. If the console outputs a message, the user should run the commands specified in the message. For example, if the output says: .. code-block:: console Please do the following to activate conda: > conda activate srw_app then the user should run |activate|. Otherwise, the user can continue with configuring the workflow. .. _AQMConfig: Configure and Experiment --------------------------- Users will need to configure their experiment by setting parameters in the ``config.yaml`` file. To start, users can copy a default experiment setting into ``config.yaml``: .. code-block:: console cd ush cp config.aqm.yaml config.yaml Users will need to change the ``MACHINE`` and ``ACCOUNT`` variables in ``config.yaml`` to match their system. They may also wish to adjust other experiment settings. For more information on each task and variable, see :numref:`Section %s `. The community AQM configuration assumes that users have :term:`HPSS` access and attempts to download the data from HPSS. However, if users have the data on their system already, they may prefer to add the following lines to ``task_get_extrn_*:`` in their ``config.yaml`` file, adjusting the file path to point to the correct data locations: .. attention:: HPSS is not available on ``gaeac6``. User-staged data must be used. .. code-block:: console task_get_extrn_ics: USE_USER_STAGED_EXTRN_FILES: true EXTRN_MDL_SOURCE_BASEDIR_ICS: /path/to/data task_get_extrn_lbcs: USE_USER_STAGED_EXTRN_FILES: true EXTRN_MDL_SOURCE_BASEDIR_LBCS: /path/to/data On Level 1 systems, users can find :term:`ICs/LBCs` in the usual :ref:`input data locations ` under ``FV3GFS/netcdf/2023111000`` and ``FV3GFS/netcdf/2023111000``. Users can also download the data required for the community experiment from the `UFS SRW App Data Bucket `__. Users may also wish to change :term:`cron`-related parameters in ``config.yaml``. In the ``config.aqm.yaml`` file, which was copied into ``config.yaml``, cron is used for automatic submission and resubmission of the workflow: .. code-block:: console workflow: USE_CRON_TO_RELAUNCH: true CRON_RELAUNCH_INTVL_MNTS: 3 This means that cron will submit the launch script every 3 minutes. Users may choose not to submit using cron or to submit at a different frequency. Note that users should create a crontab by running ``crontab -e`` the first time they use cron. When using the basic ``config.aqm.yaml`` experiment, the AQM pre-processing tasks are automatically turned on because ``"parm/wflow/aqm_prep.yaml"`` appears in the list of workflow files in the ``rocoto: tasks: taskgroups:`` section of ``config.yaml`` (see :numref:`Section %s ` for task descriptions). To turn on AQM *post*-processing tasks in the workflow, include ``"parm/wflow/aqm_post.yaml"`` in the ``rocoto: tasks: taskgroups:`` section, too (see :numref:`Section %s ` for task descriptions). .. attention:: The module required to run the post-processing tasks is available only on WCOSS2. Therefore, ``aqm_post.yaml`` should not be added to the ``rocoto: tasks: taskgroups:`` section of ``config.yaml`` on any other platforms. Generate the Workflow ------------------------ Generate the workflow: .. code-block:: console ./generate_FV3LAM_wflow.py Run the Workflow ------------------ If ``USE_CRON_TO_RELAUNCH`` is set to true in ``config.yaml`` (see :numref:`Section %s `), the workflow will run automatically. If it was set to false, users must submit the workflow manually from the experiment directory: .. code-block:: console cd ${EXPT_BASEDIR}/${EXPT_SUBDIR} ./launch_FV3LAM_wflow.sh Repeat the launch command regularly until a SUCCESS or FAILURE message appears on the terminal window. See :numref:`Section %s ` for more on the ``${EXPT_BASEDIR}`` and ``${EXPT_SUBDIR}`` variables. Users may check experiment status from the experiment directory with either of the following commands: .. code-block:: console # Check the experiment status (for cron jobs) rocotostat -w FV3LAM_wflow.xml -d FV3LAM_wflow.db -v 10 # Check the experiment status and relaunch the workflow (for manual jobs) ./launch_FV3LAM_wflow.sh; tail -n 40 log.launch_FV3LAM_wflow To see a description of each of the AQM workflow tasks, see :numref:`Section %s `. .. _AQMSuccess: Experiment Output -------------------- The workflow run is complete when all tasks display a "SUCCEEDED" message. If everything goes smoothly, users will eventually see a workflow status table similar to the following: .. code-block:: console CYCLE TASK JOBID STATE EXIT STATUS TRIES DURATION ============================================================================================ 202302170000 make_grid 47411619 SUCCEEDED 0 1 36.0 202302170000 make_orog 47411728 SUCCEEDED 0 1 151.0 202302170000 make_sfc_climo 47411801 SUCCEEDED 0 1 58.0 202302170000 nexus_gfs_sfc 47411620 SUCCEEDED 0 1 37.0 202302170000 nexus_emission_00 47411729 SUCCEEDED 0 1 251.0 202302170000 nexus_emission_01 47411730 SUCCEEDED 0 1 250.0 202302170000 nexus_emission_02 47411731 SUCCEEDED 0 1 250.0 202302170000 nexus_post_split 47412034 SUCCEEDED 0 1 44.0 202302170000 fire_emission 47411621 SUCCEEDED 0 1 19.0 202302170000 point_source 47411732 SUCCEEDED 0 1 82.0 202302170000 aqm_lbcs 47412961 SUCCEEDED 0 1 159.0 202302170000 get_extrn_ics 47411622 SUCCEEDED 0 1 314.0 202302170000 get_extrn_lbcs 47411623 SUCCEEDED 0 1 0.0 202302170000 make_ics_mem000 47659593 SUCCEEDED 0 1 126.0 202302170000 make_lbcs_mem000 47659594 SUCCEEDED 0 1 113.0 202302170000 run_fcst_mem000 47659742 SUCCEEDED 0 1 763.0 202302170000 run_post_mem000_f000 47659910 SUCCEEDED 0 1 30.0 202302170000 run_post_mem000_f001 47660029 SUCCEEDED 0 1 30.0 202302170000 run_post_mem000_f002 47660030 SUCCEEDED 0 1 31.0 ... 202302170000 run_post_mem000_f006 47660110 SUCCEEDED 0 1 29.0 ============================================================================================ 202302170600 nexus_gfs_sfc 47659421 SUCCEEDED 0 1 44.0 202302170600 nexus_emission_00 47659475 SUCCEEDED 0 1 323.0 202302170600 nexus_emission_01 47659476 SUCCEEDED 0 1 323.0 202302170600 nexus_emission_02 47659477 SUCCEEDED 0 1 329.0 202302170600 nexus_post_split 47659595 SUCCEEDED 0 1 60.0 202302170600 fire_emission 47659422 SUCCEEDED 0 1 18.0 202302170600 point_source 47659478 SUCCEEDED 0 1 128.0 202302170600 aqm_ics 47659597 SUCCEEDED 0 1 159.0 202302170600 aqm_lbcs 47659598 SUCCEEDED 0 1 158.0 202302170600 get_extrn_ics 47659423 SUCCEEDED 0 1 493.0 202302170600 get_extrn_lbcs 47659424 SUCCEEDED 0 1 536.0 202302170600 make_ics_mem000 47659594 SUCCEEDED 0 1 134.0 202302170600 make_lbcs_mem000 47659596 SUCCEEDED 0 1 112.0 202302170600 run_fcst_mem000 47659812 SUCCEEDED 0 1 1429.0 202302170600 run_post_mem000_f000 47659998 SUCCEEDED 0 1 30.0 202302170600 run_post_mem000_f001 47660042 SUCCEEDED 0 1 31.0 202302170600 run_post_mem000_f002 47660043 SUCCEEDED 0 1 29.0 ... 202302170600 run_post_mem000_f012 47660134 SUCCEEDED 0 1 30.0 .. _AQM-more-tasks: Additional Tasks for AQM =============================== Structure of SRW-AQM Workflow -------------------------------- :numref:`Figure %s ` illustrates the full non-:term:`DA ` SRW-AQM workflow using a flowchart. Compared to the uncoupled (atmosphere-only) workflow (see :numref:`Table %s `), SRW-AQM has additional tasks for pre- and post-processing. For pre-processing, multiple emissions data such as NEXUS, fire, and point-source emissions are retrieved or created for air quality modeling. Moreover, the chemical initial conditions (ICs) are extracted from the restart files of the previous cycle and added to the existing IC files. The chemical lateral boundary conditions (LBCs) and the GEFS aerosol data are also added to the existing LBC files. For post-processing, air quality forecast products for ozone (O3) and 2.5-micron particulate matter (PM2.5) are generated, and the bias-correction technique is applied to improve the accuracy of the results. .. _FlowProcAQM: .. figure:: https://github.com/ufs-community/ufs-srweather-app/wiki/WorkflowImages/SRW-AQM_workflow.png :alt: Flowchart of the SRW-AQM tasks. *Workflow Structure of SRW-AQM (non-DA)* Pre-processing Tasks of SRW-AQM ------------------------------------ The pre-processing tasks for air quality modeling (AQM) are shown in :numref:`Table %s `. They are defined in the ``parm/wflow/aqm_prep.yaml`` file. .. _TasksPrepAQM: .. list-table:: *Tasks for Pre-Processing of AQM* :widths: 20 50 :header-rows: 1 * - Task Name - Description * - nexus_gfs_sfc - Retrieves the GFS surface files from the previous cycle in near real-time (NRT) or from the current cycle in retrospective cases. The surface radiation, soil moisture, and temperature fields are needed to predict the :term:`MEGAN` biogenics emissions within the ``nexus_emission_##`` task. * - nexus_emission_## - Prepares the run directory with gridded emissions inputs, runs the :ref:`NEXUS` to create model-ready emissions for the given simulation day, and post processes NEXUS output to make it more readable. The task will also split the task into ``##`` jobs set by the user in ``config.yaml`` using the ``NUM_SPLIT_NEXUS`` variable. * - nexus_post_split - Concatenates the NEXUS emissions information into a single netCDF file (needed for the forecast) if NEXUS was split into multiple jobs using the ``NUM_SPLIT_NEXUS`` variable. * - fire_emission - Converts both satellite-retrieved gas and aerosol species emissions (:term:`RAVE`) from mass (kg) to emissions rates (kg/m2/s) and creates 3-day hourly model-ready fire emissions input files. * - point_source - Aggregates the anthropogenic point source sectors of the National Emission Inventory (NEI) into a ready-to-input point-source emission file based on the weekday/weekend/holiday patterns of each sector and the date/time of the simulation. * - aqm_ics - Creates a chemical initial conditions file by using the previous cycle restart files. * - aqm_lbcs - Adds the chemical lateral boundary conditions (LBCs) to the meteorological LBCs to form the full set of ready-to-input LBCs for the simulation. This task includes two sub-tasks: (1) addition of the gaseous species LBCs and (2) addition of dynamic aerosol LBCs. The former adds static gaseous LBCs using monthly mean global data. The latter is the parallel job, which extracts the GEFS-Aerosol Model's output along the regional domain and performs the species conversion from :term:`GOCART` aerosols to CMAQ aerosols. Post-processing Tasks of SRW-AQM ------------------------------------ The post-processing tasks for air quality modeling (AQM) are shown in :numref:`Table %s `. They are defined in the ``parm/wflow/aqm_post.yaml`` file. Since the module required to run these tasks is available only on WCOSS2, ``aqm_post.yaml`` should not be added to the ``rocoto: tasks: taskgroups:`` section of the configuration file ``config.yaml`` on other platforms. .. _TasksPostAQM: .. list-table:: Tasks for Post-processing of AQM :widths: 20 50 :header-rows: 1 * - Task name - Description * - pre_post_stat - Creates surface (i.e., model first level) meteorological and chemical files to support air quality product generation and generate training data to support bias correction tasks. * - post_stat_o3 - Generates air quality forecast products, including hourly average and statistical products, for O3 (e.g., daily 8-hour average maximum O3). * - post_stat_pm25 - This task generates air quality forecast products, including hourly average and statistical products, for PM2.5 (e.g., 24-hour average PM2.5). * - bias_correction_o3 - Applies a bias-correction technique (e.g., analog ensemble) to improve the raw model forecast for O3 and generates the bias-corrected O3 products. * - bias_correction_pm25 - Applies a bias-correction technique (e.g., analog ensemble) to improve the raw model forecast for PM2.5 and generates the bias-corrected PM2.5 products. WE2E Test for AQM ======================= Build the app for AQM: .. code-block:: console ./devbuild.sh -p=ursa -a=ATMAQ Add the WE2E test for AQM to the list file: .. code-block:: console cd /path/to/ufs-srweather-app/tests/WE2E echo "custom_ESGgrid" > my_tests.txt echo "aqm_grid_AQM_NA13km_suite_GFS_v16" >> my_tests.txt Run the WE2E test: .. code-block:: console $ ./run_we2e_tests.py -t my_tests.txt -m ursa -a gsd-fv3 -q AQM Use Cases ============= An AQM "use case" is a scientifically interesting air quality event with preconfigured SRW workflow templates. ICs, LBCs, and fixed files are also staged in the `SRW App Data Bucket `__, and can be downloaded using the `AQM-Eval Data Sync utility `__. If data is downloaded using this utility, setting ``cpl_aqm_parm.USE_AQM_S3_DATA_STAGE = true`` and ``AQM_STAGE_DST_DIR = `` will populate the root paths for data dependencies. .. list-table:: Supported Use Cases :widths: 20 10 20 :header-rows: 1 * - Name - Key - Configuration Template * - `Atmospheric Emissions and Reactions Observed from Megacities to Marine Areas `__ - :term:`AEROMMA` - ``./ush/aqm-use-cases/config.aqm.AEROMMA.yaml`` * - `Fall Ozone` - FALL_OZONE - ``./ush/aqm-use-cases/config.aqm.FALL_OZONE.yaml`` .. _acquire-use-case-data: Acquiring Use Case Data --------------------------- AQM data requirements are relatively large. Using the AEROMMA campaign as an example, expect a minimum of ~28 terabytes of storage for the full use case, with ~3 terabytes allocated to fixed files. Hence, use case configurations assume a user will want to evaluate a 48-hour forecast before extending to the recommended use case forecast window. In this situation, download the time-varying data using the data sync utility's ``--snippet`` flag. .. code-block:: console $ DST_DIR= $ conda run -n aqm-eval --no-capture-output aqm-data-sync time-varying --dst-dir ${DST_DIR} --use-case AEROMMA --snippet After testing a 48-hour forecast, remove the ``--snippet`` flag to download the full time-varying dataset, or provide a custom end date via the ``--last-cycle-date`` flag. If you are not on a Tier 1 platform and have not acquired the SRW fixed file data, also download the SRW fixed data using: .. code-block:: console $ # Use the same destination directory as the time-varying data download. $ DST_DIR= $ conda run -n aqm-eval --no-capture-output aqm-data-sync srw-fixed --dst-dir ${DST_DIR} Now, setting ``cpl_aqm_parm.USE_FIX_AQM_S3_DATA_STAGE= true`` will adjust fixed files paths to point to the ``AQM_STAGE_DST_DIR``. .. tip:: The sync utility is resumable. Re-use the destination directory if a sync operation is canceled, interrupted, or additional temporal data is needed. Synchronization is a one-way sync from the S3 source. Running a Use Case ---------------------- Once data is appropriately staged, the use case workflow configuration file may be used like any other SRW workflow configuration file. The preconfigured template assumes users have downloaded data using the data synchronization utility. If not, the user should move any use case-specific settings to the default AQM configuration file (e.g., cycle dates). Assuming the data stage is done correctly for the use case, users should generally focus on updating these configuration variables: .. list-table:: Important Use Case Configuration Variables :widths: 20 50 :header-rows: 1 * - Configuration Variable - Description * - ``user.MACHINE`` - Platform running the workflow. * - ``user.ACCOUNT`` - Account holding the core hours. * - ``cpl_aqm_parm.AQM_STAGE_DST_DIR`` - Required path to the destination (root) directory of the data synchronization operation. * - ``workflow.DATE_LAST_CYCL`` - Defaults to two 24-hour forecast cycles. * - ``cpl_aqm_parm.USE_FIX_AQM_S3_DATA_STAGE`` - Defaults to false. Set to true if fixed data was downloaded to the stage directory. MELODIES MONET (MM) Evaluation ================================ SRW-AQM provides an optional task group leveraging :term:`MELODIES MONET` (`documentation `__) for model evaluation. .. epigraph:: MELODIES MONET is a joint project between NSF NCAR and NOAA to develop a modular framework that integrates existing and future diverse atmospheric chemistry observational datasets with chemistry model results for the evaluation of air quality and atmospheric composition. -- MELODIES MONET Documentation How to Run the MM Evaluation ------------------------------ .. note:: The AQS PM and AQS VOCs evaluations do not run on Derecho. Set the packages' ``active`` flag to ``false`` (i.e., ``melodies_monet_parm.aqm.packages.aqs_pm.active: false``) when using Derecho. To run the evaluation suite, a user will need to follow the steps described in the [aqm-mm-eval](https://github.com/NOAA-EPIC/AQM-Eval/wiki/aqm%E2%80%90mm%E2%80%90eval) wiki. An example MM evaluation configuration tuned to the AEROMMA use case is provided at ``ush/aqm-use-cases/config.aqm.AEROMMA.yaml``. High-level steps required to run an evaluation using the AEROMMA use case data: 1. Download observational data using ``aqm-data-sync``: https://github.com/NOAA-EPIC/AQM-Eval/wiki/aqm%E2%80%90mm%E2%80%90eval#stage-observational-data 2. Configure observational datasets: https://github.com/NOAA-EPIC/AQM-Eval/wiki/aqm%E2%80%90mm%E2%80%90eval#observational-datasets 3. Activate the evaluation task group by setting ``melodies_monet_parm.aqm.active: true`` and uncommenting ``workflow.taskgroups[parm/wflow/aqm_post_melodies_monet.yaml]``.