OVERVIEW

Earthworm is a free, open-source, modular, and extensible waveform and automatic earthquake processing software package written primarily in the C language. Originally developed by the United States Geological Survey (USGS), Earthworm has been an open-source project since 1993, with contributors from all over the world.

In an Earthworm system, processing is performed by separate programs, called modules. Each module performs a specific task and communicates with the other Earthworm modules by writing typed, ordered message packets into shared memory regions, called rings.

In a typical Earthworm system, a device-specific module obtains digitized samples (waveform data) of the signal measured by one or more sensors attached to a device (channels), such as an A/D board or a seismograph. The module converts the data into a sequence of standard Earthworm waveform data message packets, called TRACEBUFs. A TRACEBUF identifies the source of the data it contains using a unique signature for each channel, called a SCNL. The waveform data segment it contains is identified by the data type of each sample, the number of samples, the UTC time of the first sample, and the sample rate.

The module writes each TRACEBUF, in order, into a shared memory ring for further processing by other modules. Various processing modules then operate in conjunction on the messages in one or more rings as they consume data from their upstream providers and produce processed data for their downstream consumers.

SCNL is an acronym for Station ID, Channel ID, Network Code, and Location ID. Earthworm follows the SCNL naming convention specified in the Standard for the Exchange of Earthquake Data (SEED) Reference Manual:

Station ID      1 to 5 [UN] characters  
Channel ID      3 [UN] characters  
Network Code    1 or 2 [ULN] characters  
Location ID     1 or 2 [UN] characters, or the reserved characters "--", if omitted  

[UN]  is the set of upper case and numeric ASCII characters  
[ULN] is the set of upper case, lower case, and numeric ASCII characters

Network codes are globally unique, and are assigned by the International Federation of Digital Seismograph Networks (FDSN). Each network operator assigns the remaining identifiers for use within their own network.

The Station ID refers to the particular site (station) of one or more a sensors within a network, such as a 3-component accelerometer package oriented Up-Down (Z), N-S (N), and E-W (E).

The Channel ID encodes the band (based on the sample rate), sensor type, and sensor orientation. Channel IDs are assigned following Appendix A: Channel Naming in the SEED Reference Manual.

Location IDs are optional. They are used as necessary to differentiate sensor packages deployed at the same site, but at different locations. For example, sensors placed at different depths in a borehole array. Or, sensors placed throughout a building to measure the structural response to strong ground shaking or high wind loads.

NOTE: The data source identifier specification defined by the FDSN (http://docs.fdsn.org/projects/source-identifiers/en/v1.0/) provides for significantly longer network codes, station IDs, and location IDs than does SEED format. At present, Earthworm is not compatible with the longer identifiers permitted by the FDSN; data interchange with some FDSN networks may not be possible.

Example Earthworm SCNLs for a hypothetical USGS Earthquake Early Warning (EEW) station at Zzyzx, California, with a surface sensor package (no location ID) and a downhole string of sensors at 5 m depth (location D1), 10 m depth (location D2), etc.:

  S    C  N  L    Network   Station   Location  Channel (encoded: band, sensor, orientation)
----- --- -- --   -------- --------- ---------- ---------------------------------------------
ZZYZX HNZ EW --   USGS EEW Zzyzx, CA  surface   H:200 sps, N:accelerometer,         Z:Up-Down
      HNN    --                       surface   H:200 sps, N:accelerometer,         N:N-S
      HNE    --                       surface   H:200 sps, N:accelerometer,         E:E-W
      HHZ    D1                       5 m depth H:100 sps, H:broadband seismometer, Z:Up-Down
      HHN    D1                       5 m depth H:100 sps, H:broadband seismometer, N:N-S
      HHE    D1                       5 m depth H:100 sps, H:broadband seismometer, E:E-W
      HHZ    D2                      10 m depth H:100 sps, H:broadband seismometer, Z:Up-Down
      HHN    D2                      10 m depth H:100 sps, H:broadband seismometer, N:N-S
      HHE    D2                      10 m depth H:100 sps, H:broadband seismometer, E:E-W

Earthworm includes nearly 200 modules, categorized by purpose, such as seismic processing, archiving, system control, etc. For example, if you have data from at least four stations, there are standard Earthworm seismic processing modules to associate phase arrivals and compute earthquake locations. Existing modules can be modified, or new modules written, to provide additional features.

An Earthworm system runs on a single computer. However, Earthworm provides modules for standard protocols used by regional seismic networks to share waveforms, picks, and other information, whether by exchanging data between export/import modules running on multiple Earthworm instances, or through the use of public information services, for example, by hosting web services or issuing notifications via emails, SMS texts, pagers, etc.

The USGS funds Instrumental Software Technologies, Inc. (ISTI), for maintenance, development, and distribution of new Earthworm releases. The University of Memphis and ISTI host the Earthworm documentation and distribution web sites.

DOCUMENTATION

The Earthworm documentation can be found within this distribution at doc/WEB_DOC/index.html. The documentation for the latest release can be found online at http://folkworm.ceri.memphis.edu/ew-doc/ or http://earthwormcentral.org/. The latest documentation from the master branch, updated with each commit, can be read online at https://seismic-software.gitlab.io/earthworm/.

DOWNLOAD EARTHWORM

Earthworm releases are available at http://folkworm.ceri.memphis.edu/ew-dist/ and http://earthwormcentral.com/distribution/. Pre-compiled binary distributions are available for several operating systems and hardware architectures. If a suitable binary distribution is not available for your computer, you will have to build the binaries from an Earthworm source distribution, found at the same web sites. (See COMPILING below.)

You can also clone the Earthworm GitLab repository, https://gitlab.com/seismic-software/earthworm, and build Earthworm yourself. The master branch has the latest fixes and features; tagged releases are the tested versions. (See COMPILING below.)

GETTING STARTED

Minimum Earthworm Setup

• Create the parameter and log file directories
• Find the existing Installation ID for your institution or acquire a new one
• Set the environment variables in the run script for your configuration
• Add the modules you want to use to the startstop configuration file
• Make sure all the modules you use have a unique Module ID in the earthworm.d configuration file
• Add the modules you want to monitor to the statmgr configuration file
• Configure each module's .d configuration file for startstop
• For modules being monitoried by statmgr, configure each module's .desc configuration file

The Earthworm Directory Structure

	EW_HOME/
	├── EW_VERSION/
	│   └──bin/         Earthworm executables are here
	└── run_working/
	    ├── data/       EW_DATA_DIR
	    │   ├── mseed/  E.g., miniSEED event files                                                                                    
	    │   └── heli/   E.g., pseuso-helicorder web pages
	    ├── log/        EW_LOG
	    └── params/     EW_PARAMS

Choose a suitable location for the Earthworm top-level directory on your system. The EW_HOME environment variable stores this path name. Unpack the Earthworm distribution into this directory. Every time there is a new Earthworm release, the name of the directory containing the release will change, such as earthworm_7.9, earthworm_7.10, etc. The EW_VERSION environment variable stores the directory name of the release you are using. The combination of EW_HOME and EW_VERSION determines the location of the bin directory, which stores the executables that will be run when you start Earthworm.

Also in the top-level directory is a directory containing the configuration and data files for your Earthworm instance, typically called run_working. Under the run_working directory are directories containing your Earthworm configuration (parameter) files, the log files written by the modules when you run Earthworm, and a place for data files read and written by your Earthworm modules. These directories are typically called params, log, and data, and their paths are stored in the EW_PARAMS, EW_LOG, and EW_DATA_DIR environment variables, respectively.

The Earthworm Environment

Prior to starting Earthworm, you must run or source the Earthworm environment file appropriate to your platform. Earthworm environment files are named ew_*, and are located in the environment directory:

• ew_cygwin.bash
• ew_linux.bash (x86 and ARM)
• ew_macos.sh
• ew_sol_intel.bash
• ew_sol_intel.cmd
• ew_sol_sparc.bash
• ew_sol_sparc.cmd
• ew_win.cmd

Note that ew_win.cmd is for all Microsoft Windows operating systems.

You may copy it to your params directory.

Before you run or source your ew_* file, you must make sure all the directories actually exist that are specified by EW_HOME, EW_VERSION, EW_PARAMS, EW_LOG, and EW_DATA_DIR, and that EW_INSTALLATION is properly defined.

Running Earthworm

Two key programs are the core of Earthworm: startstop and statmgr. As its name implies, startstop starts and stops Earthworm according to the instructions in its configuration file, startstop_*.d. The startstop configuration file contains the command line to run each Earthworm module, including the name of the module and its corresponding .d configuration file. statmgr monitors the health of running Earthworm modules, as configured in its own .d file. The .desc file for each module configures how the module is monitored and, if allowed, is killed and restarted, when necessary.

See the module documentation in doc/WEB_DOC/index.html for more information on configuring startstop, statmgr, and other modules.

The easiest way to get started is to run an existing Earthworm configuration, see how it works, and then modify it for your own needs. Start with the Memphis Test Suite, available from the Earthworm GitLab package registry at https://gitlab.com/api/v4/projects/seismic-software%2Fearthworm/packages/generic/memphis-test-data/latest/memphis_test.zip. All versions are listed at https://gitlab.com/seismic-software/earthworm/-/packages?search=memphis-test-data.

Read the Test Suite params/README.memphis.test file and try running it. Note that a minor difference in the depth value (i.e., 11.74 versus 11.71) is acceptable.

When the Test Suite runs properly, and the results are correct, you can request a new Installation ID from Mitch Withers at the University of Memphis. (See the instructions at the top of earthworm_global.d for how to do this. earthworm_global.d is distributed in the environment directory, but must be placed in your parameters directory to become active.) Your Installation ID is required in your .desc files as well as some .d files. You can modify the files in the Test Suite params directory to replace INST_MEMPHIS with your Installation ID, or you can get a fresh set of parameters from $EW_HOME/$EW_VERSION/params. If you modify the Test Suite files, compare them with the .d files in $EW_HOME/$EW_VERSION/params, in case any options or their descriptions have changed.

In a git checkout, the module .d and .desc files are copied into $EW_HOME/$EW_VERSION/params from the module directories under src when you build Earthworm (make unix or nmake nt in src); to copy them without building, run bin/copy_params.bash (bin\copy_params.cmd on Windows).

It is recommended that you copy the .d and .desc configuration files for the modules and other programs you need to your params directory from $EW_HOME/$EW_VERSION/params.

As you've seen from the Test Suite, type startstop to run Earthworm. (To have Earthworm automatically start, you can use a Unix startup script for /etc/init.d or you can use the StartStop Service on Windows.)

If Earthworm does not run properly, check the log files directory if the error output from startstop isn't helpful. If you need help, join and then email the Google Groups' Earthworm Community Forum.

Find Out More

If you want to learn more about Earthworm and the documentation isn't enough, or if you want to learn how to program your own Earthworm modules, ISTI offers an Earthworm class approximately yearly. Contact ISTI to obtain the current schedule, availability, and cost for the course at http://www.isti.com/contact.

COMPILING

See How To Compile for the packages each platform needs and the steps to build Earthworm on Linux, macOS, and Windows. That online copy is updated with each commit to the master branch. The same page is in your clone or source distribution at doc/WEB_DOC/PROGRAMMER/how-to-compile.html; open that copy in a browser to get the instructions that match that source.

In short, after installing the build tools, source (or, on Windows, run) the environment file for your platform from the environment directory, then from the src directory run:

• make unix — For Linux, macOS, and other POSIX systems
• nmake nt — For Windows

See src/README for troubleshooting notes.

DEVELOPER WORKFLOW ON GITLAB

Earthworm development is hosted on GitLab at http://gitlab.com/seismic-software/earthworm/:

NOTE: To create an issue you must Sign in (top-right) with a GitLab account. Creating branches and merge requests in the Earthworm project (steps 4 onward) requires Earthworm developer access; without it, fork the project and open a merge request from your fork.

After you have signed in, you will be taken to the Earthworm Project overview page:

Use the sidebar on the left to navigate the Earthworm GitLab developer web site.

1. From the Earthworm GitLab developer web site, select Issues in the sidebar to open the Issues page:
   

2. Select the New issue button at the top-right to open a New Issue form:
   

   
   Fill in the New Issue form as best you can. You must provide a Title; the other fields are optional.

   Use the Description field to describe the issue, when it occurs or how to replicate it, and any proposed remedy or workaround. The description should be detailed enough so another developer will understand the issue well enough to work on a solution.

   You can assign the issue to yourself or leave it unassigned.

3. When you have completed the New Issue form, select the Create issue button at the bottom to create your new issue:
   
   (Select Okay! to dismiss the small information window near the top of the page, if it appears.)

4. Select the Create merge request button to open a New merge request form:
   

   
   The name of the GitLab branch for your new issue appears in the From... line at the top of the New merge request form (immediately under the form title, in bold). You will supply the branch name later when you git checkout a local copy of your new branch on your own computer.

   You can assign the issue to yourself or leave it unassigned. Any reviewers you select will be notified when updates are made.

   The Merge options to Delete source branch ... and Squash commits ... are recommended.

5. After you have reviewed the New merge request form, select the Create merge request button at the bottom. This will automatically create a merge request for your new branch:
   

   You will use this merge request later after you fix the issue and git commit it to your local branch, and then git push your updated local branch to the Earthworm GitLab repository.

6. Select Project overview in the sidebar to return to the Earthworm Project overview page:
   
   Notice the number of Branches for the Earthworm project has increased.

7. Select the Clone pull-down menu button on the right, then select the Copy URL icon to the right of the Clone with HTTPS text:
   
   This will copy the Earthworm GitLab repository web address (https://gitlab.com/seismic-software/earthworm.git) to your clipboard.

8. If you already cloned the repository following doc/WEB_DOC/PROGRAMMER/how-to-compile.html, skip to step 9. Otherwise, from a terminal window, navigate to your local top-level Earthworm development directory, such as ~/earthworm, and use git clone to clone the Earthworm GitLab repository (paste the URL you just copied) to a local directory, such as earthworm-working:

   cd ~/earthworm
git clone https://gitlab.com/seismic-software/earthworm.git earthworm-working

9. From the terminal window, git checkout your new branch locally:

If you have forgotten the name of your new branch, navigate to the Earthworm GitLab Issues page and select your issue from the list to open the page describing your issue. An entry will be listed under Activity from when you created your new branch in Step 4, above.

For example, if your new branch is named *124-add-scnl-description-to-readme-txt*:

`cd earthworm-working`  
`git checkout '124-add-scnl-description-to-readme-txt'`

10. Make your changes to the source code, compile, and test that the issue is fixed.

    Type man gittutorial for a brief introduction to the basic git command-line options. The current gittutorial[7] man page can be viewed at https://git-scm.com/docs/gittutorial.

11. From a terminal window, use git diff and git status to view the changes to be committed:

    Show any changes that you’ve made but not yet added to the local Git index:

    cd ~/earthworm/earthworm-working
git status

    Add any missing files to the Git index:

    git addfile_1 file_2 ...

    Show the changed/new files ready to be committed:

    git status

    If you need to make any further adjustments, do so now, and then add any newly modified content to the Git index.

12. If the summary is correct, git commit your updates to your local branch, then git push your updated local branch to the Earthworm GitLab repository:

    git commit -m'add a comment here to describe the commit'
git push

    This will start the Earthworm GitLab build and test pipelines.

    You may encounter a remote: error, such as:

    remote: error: inflate: data stream error (too many length or distance symbols)
    remote: error: inflate returned -3

    This is likely due to a temporary condition on the GitLab server. Wait a short time and retry the git push command.

13. From the Earthworm GitLab developer web site, navigate to the Merge requests page and open the merge request you created in Step 5 from the list:
    

14. Wait for the pipelines to complete:
    

    Review the changes and verify that the pipelines have passed. If there are problems, or either you or a reviewer decide further changes are necessary, repeat Steps 10 through 13 until you and the reviewers are satisfied that the issue is fixed.

15. When you and your reviewers are satisfied with the changes, select Mark as ready to merge the changes into the Earthworm master branch.

    You or one of your reviewers can mark the merge request as ready.

    Now all your changes have been merged with the Earthworm GitLab master branch.

Release Version

1. Open Deployment/Releases.
2. Click 'New release', fill in the required fields (Tag name should be in the version format, i.e., 7.10, 7.11, 8.0, etc.), and click 'Create release'.

GitLab will run the build, test and release pipelines. The release pipeline creates a corresponding RPM in Packages and Registries/Package Registry and build/ops containers in Packages and Registries/Container Registry.

GitLab Tutorials

GitLab offers many tutorials at https://docs.gitlab.com/ee/tutorials/. The Introduction to GitLab YouTube video on Tutorials: Find your way around GitLab is a useful presentation of the GitLab development process, especially their use of the Merge Request as a container for all the information about an issue, the development branch dedicated to the issue, and ongoing updates within the branch.