CMAQ

Description

From the CMAQ Home Page: CMAQ is an active open-source development project of the U.S. EPA Atmospheric Science Modeling Division that consists of a suite of programs for conducting air quality model simulations. CMAQ is supported and distributed by the CMAS Center.

CMAQ combines current knowledge in atmospheric science and air quality modeling with multi-processor computing techniques in an open-source framework to deliver fast, technically sound estimates of ozone, particulates, toxics, and acid deposition.

Version

  • 4.7

Authorized Users

  • CIRCE account holders
  • SC account holders

Platforms

  • CIRCE cluster
  • SC cluster

Modules

CMAQ requires the following module file to run:

  • apps/cmaq/4.7

Download

You’ll need to download the appropriate packages from the Community Modeling and Analysis System (CMAS) Center. You’ll need to go to the Downloads page and create an account. Follow the directions on the CMAS web site for creating this account. Once it is created, you can log into the downloads section and make the following selections:

  1. Product: CMAQ v.4.7
  2. Type of Computer: Linux-PC
  3. Compiler: Linux Intel Fortran 90

Preliminaries

We’ll need to add the appropriate modules to our environment, download, patch and build the appropriate sources. Luckily, this is all done for us with the cmaq-install script. We begin by doing the following,

  1. Load the following environment modules:
    [user@login0 ~]$ module add apps/cmaq/4.7
  2. Then run the command:
    [user@login0 ~]$ cmaq-install

Tutorials

Tutorial 1

At this point, cctm should be ready to run the included 32 km tutorial. You can follow these instructions for running it if you wish to test your installation or get a feel for the program.

a. Rename and submit the modified script to the queue by issuing

[user@login0 ~]$ cd $M3HOME/scripts/cctm
[user@login0 ~]$ cp run.cctm run.cctm.tut_day1
[user@login0 ~]$ qsub run.cctm.tut_day1

b. Verify that the job completed by checking the contents of output.$JOB_ID number and checking the status of your job.

  • Note: Data from cctm will be output to `/scratch/$USER/cctm` rather than `$M3DATA/cctm`. This is because of a long-standing bug with automounted NFS volumes (like your home directory). The `/scratch` file system appears to handle the cctm I/O patterns correctly.

Tutorial 2

  • Note: This is directly from the [[CMAQ#Documentation CMAQ Tutorial documentation] listed below. You will need to make a few changes to the run script for the second day’s run.
    • Assuming that you have used the run script as supplied for the first day’s run, the following is a “patch” style difference in run.cctm for the second day at 32 km.
  • Note: The line numbers in this patch format will not match with your run.cctm.sge file! Use the line’s context to make the correct changes!
  • WARNING: Pay very close attention to the contents of this patch! There are subtle changes that you might miss which will cause your run to fail! I would make this easier on you, but when you get to know this program a little more, you’ll see its for your own good! You must get used to using patches.
  • Remember: Left arrow at the beginning of the line means “before the change”, right arrow means “after the change”!

Copy run.cctm.tut_day1 and modify the new file as follows:

7c7
< # Usage: run.cctm >&! cctm_e2a.log &                                  #
---
> # Usage: run.cctm >&! cctm_e2b.log &                                  #
22c22
<  set APPL     = e2a
---
>  set APPL     = e2b
42c42
<  set STDATE   = 1999183       # beginning date
---
>  set STDATE   = 1999184       # beginning date
106,107c106,107
<  set EMISfile  = e_32km_cb4.19990702.ncf
< #set EMISfile  = e_32km_cb4.19990703.ncf
---
> #set EMISfile  = e_32km_cb4.19990702.ncf
>  set EMISfile  = e_32km_cb4.19990703.ncf
114,117c114,117
< #set GC_ICpath = $OUTDIR
< #set GC_ICfile = CCTM_d1bCONC.d1b
<  set GC_ICpath = $M3DATA/icon
<  set GC_ICfile = ICON_cb4_M_32_99TUT02_profile
---
>  set GC_ICpath = $OUTDIR
>  set GC_ICfile = CCTM_e2aCONC.e2a
> #set GC_ICpath = $M3DATA/icon
> #set GC_ICfile = ICON_cb4_M_32_99TUT02_profile

Run the next day’s run by submitting the script, run.cctm.tut_day2.

Tutorial 3

After running both days of the 32 km, you are ready to run the nested 8 km.

  • Note: The same rules of thumb from the above patch file still apply!

1. You will need to recompile icon script:

42c42
<  set APPL  = e1a
---
>  set APPL  = e3a
59,60c59,60
< #set ModInpt   = ( module m3conc            $Revision; )
<  set ModInpt   = ( module profile           $Revision; )
---
>  set ModInpt   = ( module m3conc            $Revision; )
> #set ModInpt   = ( module profile           $Revision; )
62c62
< #set ModMech   = ( module mc_noop           $Revision; )
---
>  set ModMech   = ( module mc_noop           $Revision; )
64c64
<  set ModMech   = ( module radm2_to_cb4      $Revision; )
---
> #set ModMech   = ( module radm2_to_cb4      $Revision; )

Likewise you will need to modify run.icon:


7c7
< # Usage: run.icon >&! icon_e2a.log &                                  #
---
> # Usage: run.icon >&! icon_e3a.log &                                  #
23c23
<  set CFG      = e1a
---
>  set CFG      = e3a
35c35
<  setenv GRID_NAME M_32_99TUT02
---
>  setenv GRID_NAME M_08_99TUT02
39c39
<  setenv LAYER_FILE $M3DATA/mcip3/M_32_99NASH/METCRO3D_benchmark2005
---
>  setenv LAYER_FILE $M3DATA/mcip3/M_08_99NASH/METCRO3D_benchmark2005
54,55c54,55
<  set IC = profile      # use default profile file
< #set IC = m3conc       # use CMAQ CTM concentration files (nested runs)
---
> #set IC = profile      # use default profile file
>  set IC = m3conc       # use CMAQ CTM concentration files (nested runs)
115c115
<     setenv CTM_CONC_1 $M3DATA/cctm/CCTM_d1bCONC.d1b
---
>     setenv CTM_CONC_1 $M3DATA/cctm/CCTM_e2aCONC.e2a

2. You will also need to recompile bcon script:

42c42
<  set APPL  = e1a
---
>  set APPL  = e3a
57,58c57,58
< #set ModInpt   = ( module m3conc            $Revision; )
<  set ModInpt   = ( module profile           $Revision; )
---
>  set ModInpt   = ( module m3conc            $Revision; )
> #set ModInpt   = ( module profile           $Revision; )
60c60
< #set ModMech   = ( module mc_noop           $Revision; )
---
>  set ModMech   = ( module mc_noop           $Revision; )
62c62
<  set ModMech   = ( module radm2_to_cb4      $Revision; )
---
> #set ModMech   = ( module radm2_to_cb4      $Revision; )

The following are “patch” style differences in run.bcon, which must now be executed for every day of the nested, 8km runs. The first difference shows how to change the script from running the profile data to the first day of the nested data:

7c7
< # Usage: run.bcon >&! bcon_e2a.log &                                  #
---
> # Usage: run.bcon >&! bcon_e3a.log &                                  #
23c23
<  set CFG      = e1a
---
>  set CFG      = e3a
34c34
<  setenv GRID_NAME M_32_99TUT02
---
>  setenv GRID_NAME M_08_99TUT02
38c38
<  setenv LAYER_FILE $M3DATA/mcip3/M_32_99NASH/METCRO3D_benchmark2005
---
>  setenv LAYER_FILE $M3DATA/mcip3/M_08_99NASH/METCRO3D_benchmark2005
56,57c56,57
<  set BC = profile      # use default profile file
< #set BC = m3conc       # use CMAQ CTM concentration files (nested runs)
---
> #set BC = profile      # use default profile file
>  set BC = m3conc       # use CMAQ CTM concentration files (nested runs)
119c119
<     setenv CTM_CONC_1 $M3DATA/cctm/CCTM_d1bCONC.d1b
---
>     setenv CTM_CONC_1 $M3DATA/cctm/CCTM_e2aCONC.e2a

The second difference shows how to change from the first day of the nest to running the second day:

7c7
< # Usage: run.bcon >&! bcon_e3a.log &                                  #
---
> # Usage: run.bcon >&! bcon_e3b.log &                                  #
64,65c64,65
<     set DATE = 1999183  # July 02, 1999
< #   set DATE = 1999184  # July 03, 1999
---
> #   set DATE = 1999183  # July 02, 1999
>     set DATE = 1999184  # July 03, 1999
119c119
<     setenv CTM_CONC_1 $M3DATA/cctm/CCTM_e2aCONC.e2a
---
>     setenv CTM_CONC_1 $M3DATA/cctm/CCTM_e2aCONC.e2b

3. Just modify the run script. The first difference shows how to modify the run script run.cctm.tut_day2 and modify the new file as follows:

7c7
< # Usage: run.cctm >&! cctm_e2b.log &                                  #
---
> # Usage: run.cctm >&! cctm_e3a.log &                                  #
22c22
<  set APPL     = e2b
---
>  set APPL     = e3a
42c42
<  set STDATE   = 1999184       # beginning date
---
>  set STDATE   = 1999183       # beginning date
92c92
<  setenv GRID_NAME M_32_99TUT02
---
>  setenv GRID_NAME M_08_99TUT02
103c103
<  set OCEANfile = SSMASK_tut32
---
>  set OCEANfile = SSMASK_tut08
107,108c107,108
<  set EMISfile  = e_32km_cb4.19990703.ncf
< #set EMISfile  = e_08km_cb4.19990702.ncf
---
> #set EMISfile  = e_32km_cb4.19990703.ncf
>  set EMISfile  = e_08km_cb4.19990702.ncf
114,117c114,117
<  set GC_ICpath = $OUTDIR
<  set GC_ICfile = CCTM_e2aCONC.e2a
< #set GC_ICpath = $M3DATA/icon
< #set GC_ICfile = ICON_cb4_M_32_99TUT02_profile
---
> #set GC_ICpath = $OUTDIR
> #set GC_ICfile = CCTM_e2aCONC.e2a
>  set GC_ICpath = $M3DATA/icon
>  set GC_ICfile = ICON_cb4_M_08_99TUT02_1999183
120c120
<  set GC_BCfile = BCON_cb4_M_32_99TUT02_profile
---
>  set GC_BCfile = BCON_cb4_M_08_99TUT02_1999183
122c122
<  set METpath   = $M3DATA/mcip3/M_32_99NASH
---
>  set METpath   = $M3DATA/mcip3/M_08_99NASH

You should submit this new job to the queue to run. Finally, the differences from the first to the second day of the 8 km runs. Copy run.cctm.tut_8k_day1run.cctm.tut_8k_day2:

7c7
< # Usage: run.cctm >&! cctm_e3a.log &                                  #
---
> # Usage: run.cctm >&! cctm_e3b.log &                                  #
22c22
<  set APPL     = e3a
---
>  set APPL     = e3b
42c42
<  set STDATE   = 1999183       # beginning date
---
>  set STDATE   = 1999184       # beginning date
108,109c108,109
<  set EMISfile  = e_08km_cb4.19990702.ncf
< #set EMISfile  = e_08km_cb4.19990703.ncf
---
> #set EMISfile  = e_08km_cb4.19990702.ncf
>  set EMISfile  = e_08km_cb4.19990703.ncf
114,117c114,117
< #set GC_ICpath = $OUTDIR
< #set GC_ICfile = CCTM_e2aCONC.e2a
<  set GC_ICpath = $M3DATA/icon
<  set GC_ICfile = ICON_cb4_M_08_99TUT02_1999183
---
>  set GC_ICpath = $OUTDIR
>  set GC_ICfile = CCTM_e2aCONC.e3a
> #set GC_ICpath = $M3DATA/icon
> #set GC_ICfile = ICON_cb4_M_08_99TUT02_1999183
120c120
<  set GC_BCfile = BCON_cb4_M_08_99TUT02_1999183
---
>  set GC_BCfile = BCON_cb4_M_08_99TUT02_1999184

You can run the second day of the 8 km run by submitting run.cctm.sge.tut_8k_day2 to the queue.

Other CMAQ Modules

One other module, procan, is included with the CMAQ distribution. Follow these instructions to build it if needed:

  1. Change directory to $M3HOME/scripts/procan
  2. Run the build script by executing:
    [user@login0 ~]$ ./bldit.pacp &> bldit.pacp.out
  3. Check the file bldit.pacp.out in the current directory.

Documentation

Home Page, User Guides, and Manuals

More Job Information

See the following for more detailed job submission information:

Reporting Bugs

Report bugs with CMAQ to the IT Help Desk: rc-help@usf.edu