CADSWES -- Release, Software Maintenance, and Development Environment Work Summary -- Fiscal Year 2014
Phil, edit 10-22-2014

This document summarizes the maintenance work completed by CADSWES in these categories from October 1, 2013 through September 30, 2014.

1. Releases, Patches and Snapshots
2. Development Environment
3. Software Maintenance
4. Download, Installation, and Release Processes
5. Licensing
6. Regression Tests
7. Bug Fixes

(1) Releases, Patches and Snapshots

Two major releases (6.4 and 6.5) and eleven (11) patch releases (6.4.1 -- 6.4.10 and 6.5.1) of RiverWare were generated this year. The work involved in generating a patch release is very similar to a major release, except that a major release typically has an extended period dedicated to bug fixing prior to the release, and a more extensive set of release notes. Release notes may be found in the RiverWare online help.

Ten (10) snapshots of the build development area were made over the course of the fiscal year to allow users to test newly implemented features before they become part of an official RiverWare release.

RiverWare 6.4 was prereleased on October 24, 2013 and released on November 15, 2013.

RiverWare 6.5 had a sequence of prereleases in 2014 on 6-24, 7-9, 7-10, 7-29, 7-30 and was released on August 7, 2014.

The tasks involved in generating releases include:

• Organizing and writing the release notes
• Testing the release executable
• Generating an updated set of online help files in PDF format
• Managing the transition of code from the development area to the prerelease area
• Administering and updating the configuration files for building the software
• Compiling and verifying the builds
• Setting up a regression test area for release test runs
• Creating the release package
• Testing the installation
• Distributing the executable on the CADSWES web site, and announcing the release.

(2) Development Environment

The Development Environment category includes all of the associated software and tools used for developing RiverWare as well as their setup and interaction. Work completed this year in this category is presented below by major topic.

• Oracle Server
• Oracle Client 12c
• CADSWES HDB Test Database
• GIT source-code repository maintenance
• Visual Studio 2010 performance analysis tool
• GPAT / Excel Add-Ins
• Decommissioning Maelstrom (Sun Solaris machine)
• Windows Development Tools
• Windows 8.1 Development Environment

Oracle Server

The Linux machine named allegheny was virtualized to the CU cloud by Managed Services. Allegheny supported Oracle and the HDB database that CADSWES developers use to test HDB connectivity from RiverWare. Oracle was installed on the virtualized server and the HDB database was moved from the old machine. Several rounds of testing were performed before correct connectivity was established from developer's machines to the new database configuration. Connectivity from SQL Plus, Oracle SQL Developer, and RiverWare are now working correctly with the new database.

Oracle Client 12c

To function with the Visual Studio 2010 compiler, the Oracle client used in building the HDB Server needed to be updated to Oracle Client 12c. The new Oracle Client was installed on a test machine and the correct Oracle OCI and OCCI libraries and DLLs for Visual Studio 2010 were integrated into the local RiverWare development environment. The libraries and DLLs from the development machine used for the ensemble work were tarred and zipped to create both 32-bit and 64-bit packages for the win-config process that allows these to be downloaded to other machines. New Oracle DLLs were made available for incorporation into the RiverWare snapshot generation process. The HDB Server that functions to connect RiverWare to an HDB Database was rebuilt with the new environment, and once a problem with the server code was addressed, the new server was tested and functioned correctly in transferring data between RiverWare and HDB.

CADSWES HDB Test Database

Connectivity to the HDB Oracle database was not working when tested in January 2014. Debugging led to the discovery that a restart of the listener on January 5, 2014 caused a change to the service name of the database (the name had been changed months earlier, but did not take effect until the restart). Connect strings for clients connecting to the database had to be updated with the new service name.

GIT source-code repository maintenance

A file permissions problem arose with the GIT repository which contains the RiverWare source code. This problem prevented software developers from committing changes to the repository. The problem was initially resolved by reconfiguring the repository to ensure appropriate permissions for CADSWES staff. Subsequently, the problem returned. The problem was reproduced in a new test repository, and analysis determined that an improperly cloned repository on another machine and operating system was corrupting the permissions. The configuration of the cloned repository was corrected. This resolved the problem.

Visual Studio 2010 performance analysis tool

The Visual Studio 2010 performance analysis tool required an updated driver, but when the new driver was installed, the tool would abort analysis with an unhelpful error message. The problem was eventually resolved by uninstalling Symantec Endpoint Protection and replacing it with an alternative application to provide a firewall and other security measures.

GPAT / Excel Add-Ins: Tab Not Visible

In setting up tests for the GPAT plugin in RiverSMART, it was discovered that the Add-Ins tab in Excel from which GPAT is initiated was hidden on a developer’s machine. There was no way to execute GPAT from the Excel interface. After research it was discovered that Visual Studio Tools for Excel, which is a COM Excel add-in, apparently keeps the Add-In tab from being visible when it is installed. When this Visual Studio Tools add-in was removed from Excel under the administrator account, this fixed the problem.

Support Activity Tracking Tool

The support tool is a Java application used by CADSWES staff to keep track of the time spent providing user support to RiverWare users. For years it's been run on Solaris machines, the last of which will soon be decommissioned. To prepare for this, the support tool was moved to a Linux machine. A search for the Java source was unsuccessful; instead a Java decompiler was used to extract the Java source from the Java archive.

Decommissioning Maelstrom (Sun Solaris machine)

Over the years RiverWare development has migrated from Solaris to Windows and Linux, to the extent that a single Solaris machine (maelstrom) remained. Maelstrom was used for a small number of tasks. It was decided to decommission maelstrom and complete the migration from Solaris. Tasks included:

1. Deploying our Support Tool (for logging user support time) on Alamosa (linux). The support tool is written in Java and was a fairly straightforward port from Solaris to Linux.
2. Deploying "modelcomp" with xmgr -- for comparing model files and plotting differences -- on Alamosa. Attempts to find a Linux xmgr binary which could be installed were unsuccessful. Subsequent attempts to find source code which could be compiled were successful.
3. Confirming usability of Flex/Bison (parsing library) tools on Alamosa. These are used within RiverWare for processing files having particular grammars, e.g. RPL ruleset and SCT files.

Windows Development Tools

RiverWare development requires a sophisticated development environment with several third party tools and libraries. Over the years an infrastructure has been developed for configuring Windows machines for RiverWare development. The infrastructure used both CVS (a source control system) and tar files for deploying the development environment. There were two problems with the infrastructure. First, using CVS was a "chicken and the egg" situation - CVS was necessary to install CVS on a Windows machine. Second, portions of the development environment had become fragmented across three directories (bin, bin.reorg and perllib). We decided to ditch CVS, to combine the three directories into a single directory, and to jettison tools which are no longer being used. This was accomplished and greatly simplified configuring Windows machine for RiverWare development.

Related: From the beginning RiverWare Windows development has been on the C: partition, in C:\RiverWare. This seems to be counter to "best practices" which separate data from the operating system to facilitate backing up data independently from the operating system. Some RiverWare tools relied on RiverWare development being in C:\RiverWare; the tools were modified to be accepting of other partitions. (They use heuristics to locate directories which are known to be part of the RiverWare development environment).

Windows 8.1 Development Environment

We configured a Windows 8.1 machine to support RiverWare development and created a document describing the procedures for doing this on other machines. To illustrate, the following are some of the steps involved in configuring a machine for RiverWare development:

• Install the compiler (Visual Studio 2010)
• Install third party software libraries (e.g., Qt and GDAL)
• Install configuration management tools (e.g., the source code versioning application, GIT, and a custom script for conducting regression tests)
• Configure permissions for access to remote source code repositories
• Create a local copy of the source code repository

As part of this work, problems were identified with the existing script for installing RiverWare libraries and that script was significantly redesigned.

(3) Software Maintenance

The Software Maintenance category includes work required to maintain the RiverWare code that is apart from work funded to implement specific new features and new functionality. This can include, for example,  removing obsolete code, porting code to use newer versions of libraries (like Qt) or to use alternate libraries, addressing compiler warnings and memory leaks, non-project improvement to online help, or changing existing code to improve maintainability and efficiency.

• RiverWare name changes
• Power Reservoir method change
• Fixes to the Qt4 port of the SCT's series data table
• Qt4 Port of Client/Server
• HDB Server Removed / OracleConnect and HdbConnect Integrated into RiverWare

RiverWare name changes

We analyzed two bugs reports suggesting changes to RiverWare names, user-selectable category and method names in the case of bug 809, slot names in the case of bug 2971. Initially, approximately 200 categories and methods were identified as candidates for renaming, and a design was developed to allow these changes to be made without affecting existing models. The software infrastructure portion of this design was implemented and tested. The new software infrastructure was utilized to update category and method names on the Aggregate Diversion Site object to a standard format.

Ultimately, approximately 300 category and method names were modified across all engineering object types to adhere to a standard format. This included formatting all names in title case, including spaces between all words, removing terms such as category, calculation or calc from the names and in some cases making the name more descriptive of what the method does. These changes were made available in RiverWare 6.5.

Changing the name of slots is more difficult for several reasons, principle among them is that external entities with which RiverWare interacts often rely heavily on the specific slot names. For example, database schemas have been developed which incorporate current slot names, and changing those names would disrupt use of those databases. CADSWES staff concluded that while it would be useful to change the names of some slots, this would not be undertaken immediately.

... additional name change work:

• Updated reach routing category and method names: The "routingMethodCategory" has been changed to "Routing," and all method names in the category have been revised to a consistent, updated format for RiverWare 6.4. For example, the method previously called "timeLagRouting" is now "Time Lag." All existing models will update automatically upon load.

Power Reservoir method change

Functionality to allow a Plant Power Cap Fraction greater than 1 was restored. This returns functionality from RiverWare 6.3.3 before the initial fix to bug 5433.

Fixes to the Qt4 port of the SCT's series data table

A broad set of SCT bugs -- mostly resulting from the Qt3-to-Qt4 port of the SCT's series data table developed in March and April, 2013 for RiverWare 6.4 -- were fixed for RiverWare 6.4.2 and 6.5. These included:

1. The 'Double Click Data Cell Toggles Detail Rows' general configuration option wasn't functioning. Double-clicking in a detail row cell (in the aggregated SCT views) was instead starting an edit of the "clicked" detail cell.
2. In the Horizontal Timestep Aggregated View, the triangle-arrow treeview control column (for showing and hiding detail rows within a timestep aggregation block) was overly wide. This occurred only in the release (non-debug) build.
3. Unintentionally starting an incell edit, and then clicking away, caused the cell's flag to be changed to "Input" (as a result of the cell editor's text being applied to the slot value). Now, when an edit completes with the edited value matching the original value (with display precision), the edit is not applied. (Note that it would be useful for users to know that any edit can be aborted by hitting the ESC key).
4. A change to a series slot's time range as a result of running an Input DMI caused the SCT to be scrolled to the beginning of the time range. This no longer occurs.
5. Red crosshatching on slot column headers -- to indicate slots on objects for which dispatching is disabled -- was not showing up in certain Windows desktop themes. This was addressed by crosshatching all cells for such slots (in the vertical time axis views).
6. The "Go To" (-slot item) operation wasn't scrolling the SCT to that item in a sufficiently precise way -- it was only insuring that the picked slot item was visible. Now the picked slot item is scrolled to the top (or left side) of the SCT's window (depending on the SCT's current axis orientation).
7. During an incell edit, the left and right arrow keys were moving the edit cursor one character to the left or right within the text being edited. A sponsor preferred that those keys terminate the edit and move "focus" to the adjacent data cell (i.e. to the left or right).
8. In the horizontal time axis views, when an incell edit was active, the Tab key was navigating to the wrong cell: one below the cell to the right.
9. Changes to the configuration of column header content (e.g. optional display of units or the aggregation summary function) did not always cause updates to the column header display.
10. The SCT Configuration dialog's "Summary" tab gives four choices for the conditions under which timestep flag colors are shown in representative vertical "slices" of aggregation summary cells (as cell background colors). Not all choices had been implemented.
11. Notes on series slot values were not correctly being displayed as tooltips.
12. A mistake was made in the implementation of the SCT configuration lock status. It had been preventing incell edits.

Qt4 Port of Client/Server

The code for the Client/Server library was ported to Qt4 in previous work, but had not been tested with a rebuilt server until the HDB server update to use Oracle Client 12c. This testing uncovered a server crash when transferring data between RiverWare and HDB. The crash was traced to code where the socket used to transfer data was being deleted. The error had to do with attempting to send an event across threads. Based on some notes in the QT code for the socket class, the C++ delete call was changed to a deleteLater call on the base socket class. This allowed socket deletion to complete successfully when it could, and fixed the crash problem with the server.

During testing of the MRM ensemble work, it was noticed that the HDB server is getting spurious commands across the socket from the RiverWare client even though RiverWare is not sending any commands. The commands appear to be the last command that the server sent being reflected back periodically across the socket to the server. This behavior looks to have started with the port of the client/server code to Qt4, which involved significant changes to the socket implementation. Preliminary debugging and testing did not uncover any fixes to the problem, so the issue was ultimately filed and fixed as bug 5448 ("Phantom commands to HDB Server"). The solution was to move the OCCI code to communicate with HDB directly into RiverWare and eliminate the HDB server -- see the next section.

HDB Server Removed / OracleConnect and HdbConnect Integrated into RiverWare

With the port to Qt4, RiverWare and server code had to be rewritten to use Qt4 socket classes. When the HDB server was modified and rebuilt to include HDB ensemble functionality, it was noted that “phantom” commands seemed to be intermittently reflected from the RiverWare client back to the server over the socket. With testing by Reclamation that involved moving a lot of data via the ensemble functionality, the server commands would get out of sync with the RiverWare client and the server would hang. It was strongly suspected that these phantom commands were causing the out of sync problem. To address this problem, the client/server mechanism was taken out of the communication process, and the code to connect to the HDB Oracle database was incorporated directly into RiverWare as described below.

The OracleConnect Visual Studio project that was in the HDB server was moved to the RiverWare Visual Studio Solution (this contains the Oracle C++ Call Interface code that connects to HDB for the OracleConnect and HdbConnect classes, as well as the OracleError class).  The RwOracle and RwHdb classes in RiverWare were modified to contain a static instance of OracleConnect and HdbConnect, respectively, Functions in RwOracle and RwHdb were modified to call the connect code directly through these static instances. Code was added to RwOracle and RwHdb classes to process Oracle error and warning messages returned from the connection code and add these messages to the DMI handler as well as posting them as RiverWare warnings. Diagnostic messages for illuminating the interactions with HDB were in the HdbClient and OracleClient classes under the diagnostic category of ClientServer. These were moved to the RwOracle and RwHdb classes under the category of Dmi Dataset.

Compilation of the new RiverWare code with Oracle libraries produced errors with type definitions of the word “boolean”.  Investigation found that Oracle include files will typedef “boolean” as an int if it is not already defined. A windows library was later declaring it as a typedef of unsigned char creating the multiple definition error. The solution was to ensure that the Windows file wtypes.h is included before the Oracle file occi.h so that the Windows definition happens first and then Oracle does not attempt to define it.

A server timeout value could be set and saved with each HDB dataset that defined the amount of time before an error was declared if the RiverWare client did not hear back from the HDB server. This was needed for the situation where the communication link between client and server became inoperable. The HDB dataset dialog was modified to remove the widgets and handler code for setting the timeout values, as these are no longer needed with the new code arrangement. The RiverWare help was modified to delete description of the server timeout functionality.

A release build of the new code was tested to ensure that the HDB functionality works correctly. Input and output DMIs as well as input and output ensembles were run to exercise all of the methods that connect to HDB. The posting of diagnostics and error messaging were also tested to verify proper operation.

(4) Download, Installation, and Release Processes

Flexera InstallShield is the software used to create RiverWare installation files. The work over the past fiscal year related to InstallShield was as follows:

• Created new IS project files for 6.4, 6.5 and 6.6 releases (several release types, throughout the year).
• Re-structured the release file holding folders for IS project files in all three release areas (builds, prerel, and release). Removed the "src" directories (originally holds 5 common RiverWare source files). Moved release files to "win" folder for common files, to "win32" folder for 32-bit files, and to "win64" folder for 64-bit files. Updated all IS project files to link from the new locations.
• Finished the installation and the configuration of the new InstallShield version “InstallShield 2013” on machine torrent. Worked with Flexera customer support to resolve a few installation issues.
• Converted the current four main IS project files from IS 2012 version to IS 2013 version. Cleaned up and removed the old release build configurations from the new project files. Working on testing the installation of each converted project file.
• Updated the release files (units and RplUnits) to the newer version in all three release file folders for the InstallShield projects files.
• Updated the InstallShield snapshot project files with the new setup for the HDB server files. The executable file rwhdbsev.exe will no longer be built, all the related Qt and Oracle DLL files are now moved up to the main install folder. The “servers” sub-folder now only holds the DSS server executable file and its Qt DLL file.
• Completed the configuration of the four newly converted InstallShield 2013 project files (for releasing 6.5 snapshots and 6.4.x patch releases). Verified the changes and tested the installation of the setup files built. Modified the bill board splash screen, added the colorful picture (the one on the RiverWare About Overview page) to the install progress window. Fixed compiler warning 3028- cannot find the string ID “CheckBox1” from the string table. Now the builds are completed without any warnings.
• Moved our InstallShield node-locked license from torrent to danshuei; resolved some issues regarding installing as a user without administrative privileges. Tested and verified the setup on the new machine. Tested the four newly converted InstallShield 2013 project files for 6.5 snapshots and 6.4.x releases on the new build machine. The new patch releases and snapshot releases are now built with the new IS version on the new build machine.
• Solved a problem of InstallShield application crashing upon launch. This had been caused by Microsoft security update KB2962872 (MS14-037). This was addressed by installing the latest IS patch release.
• Updated the Colorado River image file new version. This is displayed displayed in dialogs at the beginning and at the end of the installation.
• Upgrading InstallShield to release IS2014:
  • Downloaded and installed the latest InstallShield 2014 version to machine danshuei.
  • Cleaned up and verified all the IS2013 project files to get them ready for converting to IS2014.
  • Set up a new directory structure for holding the new IS project files with IS2014 release. The current six main project files located in the \projects\riverware\IS\is2013-project folder (64-bit version and 32-bit version for snapshot, pre-release, and official release releases) have been copied over to the new IS2014 project folder \projects\riverware\IS\is2014-project and converted to use the new IS2014 release.
  • Configured and set up the six new project files (for creating snapshot 6.6, pre-release  6.5, and release 6.5 install files), which were converted from IS2013 to IS2014.
  • Created a testing snapshot project file for generating a testing install file with the RLM 11.1 version incorporated.
• We have started looking into an issue involving RiverWare model file associations with a RiverWare executable on machines where multiple versions of RiverWare are installed. The .mdl file type is set in the IS project file and it is set in the registry during the installation.

Other work related to download, installation, and release processes included the following:

• Completed the update of the online Install User Guide.
  • Added more details to the license file install location section based on the errors reported by users in the past.
  • Added table of contents.
  • Removed all the references and instructions that are FLEXlm license related.
  • Added Viewer license executing instructions.
  • General minor changes.
    
• Updated the internal instruction document for generating RiverWare releases.
  • Added more GIT related details and reorganizing the directory structure for holding the release files that are used by InstallShield project files.
• Archived the old CVS folders to free up disk space.
• Cleaned up and reorganized the three build and release areas for better management and to free up disk space. Updated all the release manifest files.
• Updated the Copyright year in RiverWare source code and in InstallShield.
• Cleaning up the unused or the duplicated environmental variables in all project files.
• Composed a special email file describing why RiverWare default installation is designed to have each release installed into different install directory.
• Added download information to all of the license email template files so users know that they need to obtain RiverWare and related programs from our website instead of expecting physical software delivery. Also, the license re-hosting email template now includes a reminder that only three re-hosts are allowed per license contract year.
• Set up and maintained a GIT repository for all three build areas, builds for snapshot release, prerel for pre-release release, and release for official release. These three GIT repositories contain the entire checked out source files based on their specific release version. Before creating each release, a procedure is added to update the GIT repository area so all source files are updated to the most current versions. All three IS project files have been modified to retrieve these five release text files from its GIT source area.

(5) Licensing

Ongoing licensing work:

• Generation and delivery of license files to RiverWare users. Tasks include tracking of expiring licenses, contacting users to get information; generating license files; updating the license data records; providing instructions for RiverWare download, install, and floating license server manager configuration setup; and problem solving for users.
• Maintenance of licenses for internal development environment.
• Periodic reporting of sponsor and other user licenses, including temporary licenses for users attending in-house and on-site classes.

Reprise License Manager (RLM) and related licensing work:

• Tested the licenses -- both floating and node-locked licenses -- having different expiration date between RiverWare and CPLEX license keys.
• Updated the Floating License Renewal Quick Start and User Guides.
• Added section "Updating to the Latest License Server Programs" to recommend that user upgrade to the latest license server programs when restarting the license server with their new annual license file.
• Tested RiverWare version-5.X-locked license for issuing PacifiCorp's version-locked license. Created the special user instructions for running the version-locked license.
• Modified all the email template files with updated and simplified instructions for obtaining the host IDs.
• Tested running more than two license servers on one single license server system simultaneously with two different floating license files. The result is multiple generic license server processes (rlm.exe) running on one system, but only one ISV server (cadswes.exe) is allowed on one license server system.
• Updated the License Record Excel file.
• Created a new license email template file for Viewer user. This includes information about how to avoid having the “FLEXnet License Finder” window coming up every time RiverWare is started.
• Created a new email template file describing the RiverWare application version and model file version compatibility issue.
• Updated document RiverWare License File Creation Guide. This document is for internal use only.
• Updated two special email files for providing support to users with license server problems. These email files will help speed up the process of helping users resolving their license errors. The two updated are generate-diag-debug-log-file.txt and common-error.txt, which contain the instructions for users to generate a diagnostic log file for us to debug the problem.
• Upgrading RLM to version 11.1. (As of the end of FY 2014, testing with RiverWare is underway).
  • Downloaded and installed all 3 versions (Windows 64-bit, Windows 32-bit, and Solaris) of the RLM Developer Kit.
  • Configured and built version 11.1 Reprise Developer Kit, for both 32-bit and 64-bit. Provided the required configuration files to the RiverWare tools directory for incorporated into RiverWare.
  • Updated internal document "RLM for RiverWare-Configuration and Build Guide." This document contains the instructions for downloading, installing, configuring, and building the RLM Developer Kit for RiverWare.
  • RiverWare test executables were created for 32 and 64 bit Windows with Reprise 11 libraries and DLLs. This supports testing of the new Reprise license software version for incorporation into RiverWare.
  • (As of 10-2014), We are currently testing the new version with all different license files created by the new license generating program rlmsign.exe. After the new version is confirmed working without any issue, RLM release 11.1 will be checked into the version 6.6 development build.
• After full integration with RLM version 11.1, we'll start configuring and setting up the Reprise Activation Pro. This is a license activation tool which allows us to automatically deliver licenses to our users. This will initially be used for RiverWare Viewer licenses.

Dongle (USB device) License support:

• Updated online document: Dongle License User Guide for the new dongle key available from Reprise. This included updating the instructions for license file install location and adding more SafeNet help file references.
• Updated and tested the latest version of Sentinel HASP/LDK device driver files for RiverWare dongle.  The new version of the zip files is now available from the RiverWare download webpage.

(6) Regression Tests

Regression tests are a suite of RiverWare models that were automatically run every night to look for changes in results or performance from new code that was checked in to the development area. Results of the tests were evaluated on a daily basis to identify unexpected changes in model results. The model comparisons performed in the nightly regression tests can show expected differences (for example, because a new method category may have been added). When this occurred, the regression tests had to be updated to reflect the current state of the development area so model comparisons did not fail going forward. In addition, every week, the daily history of each regression test was analyzed to determine if the run time or model size had significantly changed because of new development.

In addition to the regular regression test work, the following regression test items were addressed over the course of the year:

(7) Bug Fixes

This table enumerates the 222 RiverWare bugs closed in FY 2014 (Oct 2013 through Sep 2014).

--- (end) ---