Investigation of Software Tools for Visualization of Results from Water Resources Planning Models

Project ID: 3265
Principal Investigator: Thomas FitzHugh
Research Topic: Water Operation Models and Decision Support Systems
Funded Fiscal Years: 2015
Keywords: water resources planning models; software development; data visualization

Research Question

What database, visualization, programming, and scripting tools would work best for developing a generic data visualization interface for timeseries outputs from water resources planning models? For data handling and management, the specific focus here will be on models that use the HEC-DSS format, but much of the scoping information generated here will be relevant to models that use other data output formats too.

Need and Benefit

Water resources planning studies typically require multiple runs of complicated models which have a large number of inputs and outputs. Efficiently analyzing the results of these models can be a challenge because of the large volume of data involved and the complexity of model behavior. Innovative approaches for quickly summarizing and visualizing model outputs and determining which factors control model behavior are necessary for effective use of such models in planning studies. Existing software is generally limited in terms of one or more of the following factors: (1) the volume of data that can be efficiently handled, (2) availability of visualization options relevant to water resources analysis, and (3) availability of a scripting language for customization and control diagnostics. High end database and visualization tools can also be cost prohibitive and proprietary, which limits their usefulness in situations where model results need to be distributed.

Hence there is a need for a generic database and visualization package with scripting capabilities for customization that can used to analyze results from numerous different water resources planning models. The intended package users are technical modeling staff who would use the tool during model development and results analysis. While the specific outputs from water resources planning models vary depending on model type (water operations, surface water hydrology, water quality), it is believed that a common visualization framework can be developed that is both user-friendly and applicable to results from all these models, allowing not only visualization of results from individual models but also integration of results from different models.

In terms of data handling and management, a specific focus of this effort will be on models that use the HEC-DSS format. Examples of models used by Reclamation that use HEC-DSS for data management are CalSim, CalLite, HEC-5Q, and Riverware. Speedy access to data in HEC-DSS format is challenging because of inherit limitations in its underlying libraries, hence part of the scoping conducted here will be to evaluate open source alternatives to HEC-DSS such as NetCDF and SQLite. Developing alternatives to HEC-DSS will be of interest to analysts throughout Reclamation and to our partner agencies and stakeholders.

Contributing Partners

Contact the Principal Investigator for information about partners.

Research Products

Bureau of Reclamation Review

The following documents were reviewed by experts in fields relating to this project's study and findings. The results were determined to be achieved using valid means.

Investigation of Software Tools for Visualization of Results from Water Resources Planning Models (final, PDF, 587KB)
By David O'Connor
Publication completed on September 30, 2015

The focus of this scoping study was preliminary testing of combinations of open source programming languages with open source database formats using Mid-Pacific Region river temperature model output as example model output, to estimate the time and resources needed for more in depth open source data visualization package testing and water resource tool development. The native model output format used, HEC-DSS, (U.S. Army Corps of Engineers' Hydrologic Engineering Center Data Storage System) is typical to water resource models. The first project step involved development of tools to efficiently convert data in HEC-DSS format into open source database formats (HEC-DSS source code is proprietary). Popular open source database formats NetCDF and SQLite were chosen based on project research and team experience, and we accessed and processed the data for data visualization testing via the popular open source programming languages R and Python, each of which contain open source, robust packages for data visualization. Tools to convert HEC-DSS format data to NetCDF and SQLite formats were developed and are provided and documented here: https://github.com/usbr/convertdss. In addition, USBR's Pisces data visualization application was updated to read data in SQLite format (http://www.usbr.gov/pn/hydromet/pisces/). The second project step involved data analysis and visualization using R and Python with data in NetCDF and SQLite format. All combinations yielded promising results, for both database and data visualization performance, especially considering the availability of open source interactive interfaces (''data dashboards'') that could likely be easily paired with R and Python scripting. Follow up project (USBR S&T: 4878) was not awarded funding in 2016.

Return to Research Projects

Last Updated: 6/22/20