omega - text based user interface for the OMEGA System.
omega [-options] [-parameters] [-actions]
omega is a text based user interface utility for the OMEGA System.
It can perform nearly all of the tasks involved with setting up and running a single OMEGA simulation or a series of operational simulations.
omega is used in one of three modes: Command Line - input is given as arguments after omega. Shell - input is supplied in a shell environment. Table - input is read from an ASCII text file.
omega has three groups of controls: Options - change the behavior of the program itself. Parameters - user defined choices of a simulation. Actions - do actual work of building a simulation.
In the usage examples below, the '$' is the normal UNIX csh or ksh prompt. The 'omega>' is the omega Shell Mode prompt. Only the program calling sequence is shown. No actual output from omega is shown.
In the Command Line mode Options, Parameters, & Actions are all specified on the command line using a dash (-) followed by arguments the control my take:
$ omega -caseid mycase -runid myrun -archive
The Parameter 'caseid' is set with the argument 'mycase'. The Parameter 'runmode' is set with the argument 'ops'. The Action '-new_run' is used and does not take any arguments.
The Command Line Mode is designed for single use operations. It is also the mode used when omega is called from the OMEGA System's graphical user interface, XOMEGA.
If any errors are encountered omega will stop processing the commands and exit with the proper error code.
The Shell Mode places the user in a Unix-shell-like environment where Options, Parameters, & Actions are entered one at a time. Commands entered in Shell Mode do not need a leading dash (-):
$ omega -shell
omega> caseid mycase
omega> runid myrun
omega> archive
The first input line sets the Option 'mail' with 'user@host'. The next input line sets the Parameter 'caseid' with the value 'mycase'. Input line 3 sets the Parameter 'runid' with the value 'myrun'. The last input line calls the Action 'archive'.
Shell Mode is designed to be a text based user interface to the OMEGA System.
If any errors are encountered omega will report the error and continue in the shell mode.
For the Table Mode an ASCII text file must be created that has all the Options, Parameters, & Actions listed in specific format.
$ cat /omega/ops/mycase/table
# omega ops table
1: caseid=mycase
runmode=ops opsfcst=72 remove_prior=3
iadvorder=2 hfilter=0.002 tsmooth=0.05
yesmic=true yesstat=true yessst=true
yesadm=true admsize=15
model=mrf sfcobs=none raobs=none mode=ops
adm2for=true
numproc=1 timings=true
actions=new_run
# end of table
The file name is supplied to omega as an argument on the command line
$ omega -table mycase.table
Or more often via the cron program:
$ crontab -l
# Sample crontab for use with omega
0 0 * * * /omega/bin/omega -table /omega/ops/mycase/table
# end sample crontab
In an omega table file all blank lines are ignored. All comment lines must start with a '#' character. Comment lines are ignored.
Entry lines must start with an integer number (0-99) followed by the ':' character. The number serves as the identifier (opsid) for the entry and must be unique. The ordering of the entry lines in the table file does not matter. omega will sort the entries based on the opsid number and perform them in the sorted order (e.g. 0, then 1, then 2, and so on).
In the example above there are several comment lines. The first non-comment line sets the opsid to '1'. The next few lines set some common Parameters to various values. The second to last line calls the Action 'new_run'. The last line is another comment line.
On any non-comment line, data is entered with a set of space separated fields of Options, Parameters, and Actions. Each field is composed of a keyword - value pair separated by the '=' character.
Options are entered using the option name, the '=' character, and any arguments the Option requires. A '=' is still required even if the Option does not have any arguments. Arguments are given as a comma separated list. In general once an Option is set it will remain set for subsequent entries. Use the 'no' form of the Option to turn it off. Parameters are specified with the parameter name (caseid, start, etc.), the '=' character, and the value of the Parameter.
Actions are specified with the special key 'actions', the '=' character, and then a comma separated list of the omega Actions to be performed. The Actions will be called in the order they are listed (left to right).
Table Mode is designed to be used when there is an ordered sequence of operations to perform.
If any errors are encountered omega will stop processing the table and exit with the proper error code.
omega is used to operate on an OMEGA simulation. There are three groups of controls used by omega: Options, Parameters and Actions.
The behavior of the omega program itself may be modified with Options. Most Options do not take arguments, but some have one or more arguments.
Parameters are used to set the particulars of the OMEGA Simulation (caseid, runid, start, end, etc.). Parameters are used with only one argument.
Actions do the actual work (creation, reading, writing, etc.) of building an OMEGA simulation. Actions do not use arguments, but most require that specific Parameters be set before the Action is called.
The details of all the Options, Parameters, and Actions are given below.
Options change the behavior of the omega program itself. Most are used with the option name only. Most Options do not take any arguments. Some Options may have arguments and list the format in [], and if the Option has a default it is listed in (). Some Options must have arguments and list the format in {}.
Turn ON or OFF debugging information. In Shell Mode or Table Mode use 'nodebug' to turn off debugging.
Display simple help message.
Use local time (not UTC) for creation time stamps.
Turn ON or OFF mailing of fatal error messages. If no address are supplied the default address is: $USER@$LOCALHOST.
Special Option used to start omega in Shell Mode. The only Options that may be used with shell are debug and mail. All Parameters may be used with shell. Any Actions supplied with shell will be discarded.
Special Option used to start omega in the Table Mode. An argument is required. Arguments may take the form of absolute or relative pathnames to a table file. If the pathname has ``RANDOM_CASE'' as the final element, then the preceding directory is used as the Parameter opsdir, and a random case from that directory is selected. Information in the table file has precedence over any Options or Parameters supplied on the command line.
When used with the new_run Action, the timings Option will cause omega to write out a time report file named by case and run: ``mycase_myrun.times'' in the ops directory. The format of the times file is a series of entries showing each processing step name, start time (ISO 8601), end time (ISO 8601), and wall clock time elapsed (hh:mm:ss). Note: IS0 8601 format is: YYYY-MM-DD hh:mm:ss (TZ)
Display version message.
Parameters are the particular choices and settings of the OMEGA Simulation being operated on. All Parameters must be called with a single argument. The format of the argument is listed after the Parameter name in {} Parameters that have default values are given after the format section in ().
Set the location of the OMEGA System distribution directory. The Parameter is the location of where the OMEGA System is installed. If not set by the user, omega will attempt to set it using the environment variable OMEGA. If that variable is not set, then omega uses the value given at installation.
Set the operations directory. This Parameter is the parent directory that contains the OMEGA Case directories. If not set omega will attempt to use the environment variables OMEGAOPS and OMEGA_OPS to set it. If neither of those variables are set, then omega uses the current working directory as the opsdir.
Set the caseid. A caseid runid must be 15 characters or less, and contain no white-space characters (space, tab, line-feed, etc.)
Set the runid. A runid must be 15 characters or less, and contain no white-space characters (space, tab, linefeed, etc.). See runmode below for special automatic naming conventions.
{runmode name: normal, reanalysis, restart, ops} (normal)
Set the runmode the Simulation is to be run in. The runmode of an OMEGA Simulation controls the basic linkage between the current run and other runs in the same case. Each of the major runmodes is detailed below:
normal: Normal runmode has no linkage. The current run is independent of any other runs.
ops: Daily runmode is used to generate a run based on the current day. Names are of the from YYYYMMDDhh. No other special processing is done.
reanalysis: Reanalysis runmode links the current run to a prior OMEGA run by using the prior OMEGA run's forecast as input to the initial conditions for the current run.
restart: Restart runmode is a continuation of an already existing run.
ops: Dailyops runmode is builds a simulation using a matrix of adm source locations and includes several processing programs. Need to set: opsfcst, admsize, adminject, etc. Names are of the from YYYYMMDDhh.
Set the number of processors to use for running OMEGA System Components. If the OMEGA System is installed on a multiprocessor machine, then this Parameter controls how may processors to allocate for the current case/run operation. A value of 1 will allow for serial execution of the omegamodel Component. A value greater than one will allow for parallel execution of the omegamodel Component. =item start {YYYYMMDDhh}
Set the start time. The time supplied must conform to the standard listed: four digits for (Y)ear, two digits for (M)onth, two digits for (D)ay and, two digits for (h)our.
Set the end time. The time supplied must conform to the standard listed: four digits for (Y)ear, two digits for (M)onth, two digits for (D)ay and, two digits for (h)our.
Set the met data source model. For a list of valid sources please see '$OMEGA/etc/config/met.cfg'
Set the met data source format for surface observations.
Set the met data source for]at for upper air observations.
Set the mode of fetching to either ops (mixed analysis and forecast files) or pure fcst (pure forecast files).
Set the base path to the metdata.
Set the order of the advection algorithm (1 or 2)
Parameter governing the magnitude of horizontal filtering ( < 1.0 )
Set the time smoothing parameter (<0.5)
Set the maximum time step allowed ( < 48.0)
Set the time step restriction factor (0.8 to 2.0)
Set the frequency for OMEGA output. Value must be in hours.
Turn on (true) or off (false) Dynamic Adaptation.
Set the frequency for OMEGA to adapt the grid. Value must be in hours.
Turn on (true) or off (false) Atmospheric Dispersion Model. (Assumes an omega.adm file exists in the case directory)
Output ADM files in packed binary form.
Turn on (true) or off (false) Statistic file production.
Turn on (true) or off (false) SST ingest at initialization.
Value of temperature boost (deg K) to SST input.
Turn on (true) or off (false) Concentration Model.
Turn on (true) or off (false) Microphysics Calculations.
Turn on (true) or off (false) Ocean Model.
Turn on (true) or off (false) Two way interaction with Ocean Model.
Set the frequency for calling the Ocean Model POM.
Set the frequency for saving the Ocean Model POM output.
Value used for injection interval for ADM source points used when runmode is ops.
Size of the ADM source point matrix (nxn) used when runmode is ops.
Hours to forecast when runmode is ops.
Cycle to forecast from.
{true/false} (true for runmode=ops; false for others)
If set to true will call adm processing programs adm2for.
If the simulation being created is to be run on a remote machine, omega is able to FTP the required files using a support program called mirror. This Parameter is the name (ip number: 0.0.0.0 or fully qualified domain name: abc.def.ghi) of the remote machine to run on.
Set the remote account to FTP into.
Set the remote password of the remote user. This string is passed directly to mirror and not saved or written any place. It is highly recommended that this Parameter be set only from a table file with permissions: -rw-------
Set the location of the remote case.
Minute value to use when adding an entry into the user's crontable.
Hour value to use when adding an entry into the user's crontable.
When used with the new_run Action, the remove_prior Parameter will cause omega to remove old ``past days''' runs of a case prior to creating the new ``today's'' run for the case. The argument to remove_prior must be an integer number of days and sets a date prior to ``today''. For example if today is 2000-03-26, then an argument of 2 would generate the day 2000-03-24. This date then becomes the cutoff date for old runs in the current case. Any runs whose runid is older than the cutoff date will be removed. omega will search the case directory for runs whose standard names are equal to or older than this date.
For example if today is 2000-03-26, remove_prior's argument is a 2, and the case has the following runs: 2000030100, 2000032300, 2000032400, 2000032500,
then before making ``today's'' new run (2000032600) the runs named: 2000030100, 2000032300, 2000032400, will be removed. After ``today's'' run is created, the case will contain only: 2000032500, and 2000032500 (``today's'' run and ``yesterday's'' run).
Another way to think of remove_prior's argument is ``the number of days to keep, including the run about to be created''. In the example above 2 days will remain on disk after the new run is created.
In the example, notice that the very old run named 2000030100, was also removed. omega scans the runs named with dates and remove ALL that are older than the cutoff date. When using remove_prior, omega only looks at runs that use the standard names: YYYYMMDDhh. Runs with non-date names that are older than the cutoff date are not removed.
When runmode is set to ops, this Parameter should be used to conserve disk space.
Actions do the actual work (information retrieval, file and directory creation, etc.) on the OMEGA Simulation. None of the Actions use arguments; all specific information is set by the Parameters or by system defaults.
In the list of Actions below, each description ends with a list of all the Simulation Parameters the Action requires prior to calling. If any of these Parameters are not set, then the Action will terminate with an error. If no Parameters are listed, then the Action does not require any.
Archive the specified case and run. A tar file will be created named 'caseid_runid.tar' and will contain all essential files of the case and run.
archive requires: caseid runid
Fetch_met will read the file 'caseid/runid/prep/met.inp' and call the 'fetch.csh' program for each of the met segments listed in the file. If any errors are encountered omega will go to the next segment.
fetch_met requires: caseid runid
gridgen will call the OMEGA System program 'gridgen' to produce an OMEGA grid file. The input to 'gridgen' is the file 'caseid/grid/grid.inp'. The output of 'gridgen' is are the files 'caseid/grid/grid.grd' and 'caseid/grid/grid.srf'. When a Run is created, these files will be copied to files 'caseid/runid/YYYYMMDDhh.grd' and 'caseid/runid/YYYYMMDDhh.srf'.
gridgen requires: caseid
Will return a formatted table of all cases found in the current directory.
Will return a formatted table of all the runs found in the specified case.
list_runs requires: caseid
Will return a formatted table of all the times found in the specified case and run.
list_times requires: caseid runid
This command will perform the series of tasks required to setup and run an OMEGA Simulation for the specified caseid, runid, using the runmode to to determine how to build the simulation.
new_run requires: caseid runid runmode
This command will run the omegamodel for the case and run supplied. Use numproc to specify the number of processors.
new_run requires: caseid runid runmode numproc
preprocessor will read the file 'caseid/runid/prep/met.inp' and call the 'runprep' program for each of the met segments listed in the file. If any errors are encountered omega will go to the next segment.
preprocessor requires: caseid runid
remove_run will remove the specified Run from the disk. No confirmation is given. The Case is NOT removed.
remove_run requires: caseid runid
write_opstable will create an ops table based on the current settings of the parameters. This actions is used in conjunction with cron_add.
write_opstable requires: opsdir caseid
cron_add will add an entry into the user's crontable to run the specified Case at the specified minute and hour using run_mode ops and Action new_run. A typical entry looks like (will be all one continuous line): 0 2 * * * /opt/omega/bin/omega -caseid mycase -table table
cron_add requires: caseid cron_min cron_hour
cron_del will delete (comment out) the entry from the users cron file corresponding to the specified Case, minute and hour.
This variable must be set to point to where omega/ was installed. For example if the OMEGA System was placed in /opt/omega, then the OMEGA environment variable must be set to '/opt/omega'.
The entry '$OMEGA/bin' must be early in the PATH to ensure OMEGA System version of common utilities (date, perl, ...) are used.
Either of these variables may be set to specify the operations directory. If neither are used, and the Parameter opsdir is not set, then the current working directory is used.
Temporary error file used if mail Option is set.
Temporary crontab files used if cron_add or cron_del Actions are used.
When Who What, Why & How (version #)
---------- --- -----------------------------------------------
1998 02 22 MT Initial Programming
1998 03 02 MT Added grid, time listings
1998 09 28 MT Improvements for OMEGA System 3.7.0
1998 09 29 MT Added Archive
1998 10 30 MT incorporated fetchmet and prepmet into omega
1998 11 05 MT Cleanup most functions
1998 11 09 MT Improved documentation
1998 11 10 MT Improved command line processing; fixed debugging
1998 11 11 MT Added mail function
1998 11 12 MT Reordered functions
1998 11 13 MT Streamlined output messages
1998 11 24 MT Added new_run ; fixed call_runprep to return on seg 1 errors
1998 11 30 MT Cleanup of documentation and code
1998 12 03 MT Bug fix in calling timegm
1998 12 07 MT Changed category 'Functions' to 'Actions'
Added Table mode and support subs.
1998 12 08 MT Adjusted Operations Modes
1998 12 09 MT Cleaned up Operations Modes and Options
1998 12 10 MT Added Fetch_mrfu and adjusted Call_fetch
1998 12 11 MT Added Fetch_reanalysis_* and adjusted Call_fetch
Added runmode concept to reduce specialized functions
Changed new_run to new_run
1998 12 14 MT Added SCIPUFF obs formats for reanalysis_* sources
Fixed use of copy() to avoid '.' as dest file name
1998 12 17 MT Added ADM file copies
1998 12 18 MT Adjusted dir for .sfc and .prf to use YYYYMMDD00 for all hours
Use .adr files only with runmode reanalysis_omega
Fixed parameters that get appended to bcfile
1998 12 21 MT Changed subroutine names from 'full' to 'yyyymmddhh'
1998 12 23 MT Added .bst for fetch_omega
1999 01 07 MT Fixed create_met for single segment cases
Added call_mirror and associated parameters
1999 02 01 MT Cleanup some error codes for fetch.csh (3.8.9)
1999 02 04 MT Added remove_run (3.9.0)
1999 02 10 MT Added cron functions (3.9.1)
1999 02 11 MT Added opsdir as a Parameter (3.9.2)
1999 02 11 MT Added omegadir as a Parameter (3.9.3)
Bug with global Parameter hash
1999 02 12 MT Better error linkages to xomega (3.9.4)
1999 02 16 MT Added update to ENV{PATH} with $OMEGA/bin (3.9.5)
Completed error linkages to xomega (3.9.6)
1999 02 20 MT Added gridgen, call_gridgen (4.0.0)
1999 02 26 MT Clean up on getcase, getgrid
1999 02 26 MT Added gettimes and list_times
1999 04 09 MT Added metpath to match xomega-4.0.4 (4.0.4)
also adjusted $omega to use ENV at declaration so metpath gets
a reasonable default. (4.0.5)
1999 04 29 MT Cleanup on code and doc (4.0.6)
1999 05 11 MT Mods for TVO (4.1.0):
Added Parameters: runmode ops, admsize, adminject,
opsfcst
1999 05 12 MT Added post processing abilities (4.2.0)
1999 05 19 MT Added timing report option (4.2.1)
1999 05 20 MT Added remove_prior parameter and support (4.2.2)
1999 05 24 MT Upgrade to OMEGA System 4.3 (4.3.0)
1999 06 28 MT Upgrade to OMEGA System 4.9 (4.9.0)
1999 06 28 MT Added error code 9 for installation errors (4.9.0)
1999 07 11 MT Bug fix in post_processorts for new gard.dat (4.9.1)
1999 08 13 MT Added MDS fetch (4.9.3)
1999 08 16 MT Added MDS prep
1999 08 26 MT Adjustments for preprocessor-3.7.2 (3.9.5)
1999 09 01 MT Adjustments for omegamodel-4.9.9 (4.9.9)
Added numproc
1999 09 07 MT Adjustments for using existing prf file as model source (4.9.9)
1999 09 09 MT And for prf's and sfc's for raobs and sfcobs
1999 09 17 MT Added yesstat for stat production (4.9.9.4)
1999 09 18 MT Added Parameters: opscycle (4.9.9.5)
1999 09 21 MT Added MDS as data source (model, raobs, sfcobs) (4.9.9.6)
1999 09 21 MT Added dtfactor as Parameter (4.9.9.7)
1999 10 06 MT Moved call_post to call_regard_traject (4.9.9.10)
1999 10 11 MT Adjusted defaults for opsdir, table mode, timings (4.9.10.0)
1999 10 14 MT Adjusted opstable_read, to allow for multi-line (4.9.10.1)
1999 10 19 MT Adjusted call_runprep to use List of BC times (4.9.10.2)
1999 10 21 MT Cleaned up fetch_mds and sleep calls (4.9.10.3)
1999 10 27 MT Added random case selection to table mode
Removed fetch_local subs (4.9.10.4)
1999 10 28 MT Added call to surfgen in new_run sequence (4.9.10.5)
1999 10 29 MT Added restart_run Action
Fixed remove_runs to work for all runmodes (4.9.10.6)
1999 11 01 MT Fixed create_adm sub to work with restart (4.9.10.7)
1999 11 09 MT Fixed list_cases (4.9.10.8)
1999 11 12 MT Cleanup on code and doc (4.9.10.9)
1999 12 30 MT Bug fix with parse_mds sorting times (4.9.10.11)
1999 12 30 MT Bug fix with call_runprep spliiting .BC files (4.9.10.12)
2000 01 05 MT Syntax bug fixes (4.9.10.13)
2000 01 05 MT Added call_adm_process, adm_process (4.9.10.14)
2000 02 07 MT Finalizes adm post processing (4.9.11)
2000 02 11 MT Bug fixes adm post processing (4.9.11.1)
2000 02 11 MT Bug fixes adm post processing (4.9.11.2)
2000 04 12 MT Clean up of timings; More adm processing (4.9.14)
2000 04 15 MT Improved archive_run (4.9.15)
2000 04 17 MT Fixed archive_run bugs; Cleaned up links to copies (4.9.16)
2000 04 25 MT General Clean up; Bug fix on gard.dat creation (4.9.17)
2000 05 08 MT Adjusted archive prints; (4.9.17)
2000 05 19 MT Added adm pkb processing (4.9.18)
2000 06 28 MT Bug fixes; PIDC R3; Added fetching mode (5.0.3)
Improvements in cron_add; Added write_opstable
2000 08 02 MT General Cleanup and bug fix
2000 08 03 MT Added timing print to stdout; switched to compress in archive
2000 08 11 MT Fixed archive bug; Fixed MDS directory name bug
2000 08 13 MT Changed create_adm to not clobber existing omega.adm file
2000 08 16 MT Added oldouts; Added (5.0.4)
2000 09 05 MT Code clean up (5.0.6)
2000 09 05 MT Added mrffullres; (5.1.0)
2000 09 06 MT Added sfcobs and raobds (5.1.1)
2000 09 07 RAS Added yessst, iadvorder, hfilter, tsmooth, dsst (5.1.2)
2000 09 20 MT Added sst, ocean stuff, (5.1.2)
2000 10 02 MT adjusted fetch names to match met
2000 10 04 MT clean up on reanalysis stuff (5.1.3)
2000 10 06 MT bug fixes (5.1.3)
Need to add more functionality to support other OMEGA System Components. Need to fix debugging and table, debugging and shell ON / OFF errors Need to add all files to FILES section of man page IN archive_run: goups 10 and 13 both have *.grd: so compress will give errors.
Mark Turner <turner@apo.saic.com>
This software was designed, developed and is Copyright (C) 1999 by: Science Applications International Corporation (SAIC) Center for Atmospheric Physics 1710 Goodridge Drive McLean, VA 22102