Details about the History DB
This chapter is intended for advanced users and describes the data points and internal processes during archiving. By default, WinCC OA starts the History DB (see useValueArchive).
Configuration of the value archiving
The historical archiving of value changes is executed, up to now, with the config _archive (see _archive configs).
An archiving can be set not only at the leaf level of the DP structure but for each node. Thus, also all elements below the node are automatically archived.
archiveReqTimeout
This setting is used to ensure that, if an archive is stopped, running queries will still be answered "sometime" (that is, after maximum xxx seconds). With the config entry you define how many seconds the Data Manager (=History DB) should wait (maximum) for these answers from archives (with dpPeriodRequest(), dpGetAsynch() and dpQuery()). If no answer has been received from all the archives during this period, the query will be terminated with an error. The value should remain 0 (default value), meaning unlimited waiting time. Otherwise, values beginning from 120 [secs] would be meaningful as some queries can take longer.
[data]
archiveReqTimeout = 120
-report 11
This command line option provides you with information about how many queries are requested and in what archive the individual data points are saved. The command line option also provides information about the data point name, the data point number and the time list. You can obtain general information about the History DB with the command line option -report all. Note that this command line option has to be activated for the Data Manager.
Start / shut down the archive process
The archive processes are started and shut down by the console. Hereby, the archive number ("-num x") is also defined as a command line parameter. The archive connects the Data Manager and receives the necessary configuration data (for example, assigned directory). The process name in the console is WCCOAvalarch (see Start archival).
Since parts of the archive configuration data are also stored in the archive itself, the values in the archive are compared to the values saved in the archive DP. In case of differences, the system acts as if the user would have changed the archive parameters. How to handle these kind of differences is described in the chapters describing the individual archive parameters.
If an internal archive DP is not available when an archive is started, the history level terminates the archive again. All archive parameters that are transferred via a dpSet() are also modified together in the archive.
Archive name and number
The archive name or archive number are assigned through the data point element .(_ValueArchive_x.general.arName). At the moment, the data point names for the the internal DPs are predefined as _ValueArchive_x (x represents the archive number). For redundant projects, another internal DP exists for each archive. The name of the data point ends with "_<num>_2". The figure 2 indicates the redundancy and the first figure the archive number.
It is recommended to make the settings for the data points with the panels.
In addition, there is a new element "UserValueArchive" of type bool in the internal data point of the Data Manager ("_DataManager"). This element shows if the history database is used at the moment. The value of this element is derived from the corresponding entry in the config file (see above). To establish a new archive, a new Archive DP is created and an entry must be set in the process list of the console. The value can only be read and not modified.
In addition, the element in the internal archive DPE (_ValueArchive.general.createArchive) must be set so that the archive creates the first archive record. The corresponding archive will not be started automatically.
Deleting an archive includes setting the archive DP element "state" (see below) to the state deleted and removing the archive process from the process list of the console (this is not executed automatically). When you start an archive with the status "deleted", it will be terminated immediately again. In addition, all the DPEs that are assigned to this archive must be moved to a different archive or archiving must be deactivated. This also is not executed automatically. If a DPE is assigned to a deleted archive, it will be saved in the default archive (archive 0). All the historical data of the DPEs (for the deleted archive) will be lost. The archive files will not automatically be deleted.
Monitoring and control of the archive processes
Set the archive parameters
The archive parameters are managed as elements of the archive data point (_ValueArchiv). Hereby, there are genuine parameters and actions. Whereas the parameters are only reported to the archive, actions trigger processes in the archive process. The processes are reported back through other DPEs of the archive data point. The "simple" archive parameters are described in this chapter.
The <DPE>.Get and <DPE>.Set elements are used for parameters that are not immediately modified in the archive. A required change is entered in <DPE>.Set. The current status of the archive will be displayed in <DPE>.Get.
The time data for triggering different events is implemented as references on a _TimedFunc data point type. If a DP has not yet been created and configured for a process, an error message will be shown.
Table: Parts of the DP _ValueArchive
| Element | Element | Int | Purpose | |
|---|---|---|---|---|
| arNr | 21 | archive number | ||
| state2) | 21 | 0 = stop, 1 = online, 2 = swapped out, 3 = deleted - | ||
| general | 1 | |||
| arType1) | 21 | 0 = values, 1 = messages; default: 0 | ||
| arName | 42 | name of an archive [18]; Attention: Langtext!; default: ValueArchive_x | ||
| hostNameGet3) | 25 | Host computer for .Archive [32] | ||
| hostNameSet | 25 | Set the host computer | ||
| filePathGet3) | 25 | basis path for archive records, 128], default.: <proj_path>/db/VA_000x | ||
| filePathSet | 25 | Set the basis path | ||
| fileNameGet4) | 25 | file name rule for archive records, 64],- default: YYYYMMDDHHMOSS | ||
| fileNameSet | 25 | Set file name rule | ||
| createArchive3) | 21 | 1=create archive (is only considered if the archive path does not contain any archive records yet ) | ||
| forwardToRDB | Defines whether the values are forwardedfrom the Value Archive to the RDB or not. Default: false | |||
| error1) | 1 | |||
| errNum | 21 | Error of the last archive operation. | ||
| errDp | 27 | DpId that triggered an error (for example, archive overflow) | ||
| errBit | 23 | is set in case of an archive overflow (alert handling) | ||
| size | 1 | |||
| maxDpElGet4) | 20 | Max. number of DP elements | ||
| maxDpElSet | 20 | Set the max. number of DPEs | ||
| maxValuesGet4) | 20 | Max. number of value entries | ||
| maxValuesSet | 20 | Set max. number of value entries | ||
| maxFillPctGet4) | 21 | Max. fill rate in %, 0 = 100 -1 = no change of archive record if maxValues has been reached | ||
| maxFillPctSet | 21 | Set max. fill rate | ||
| maxHeapSizeGet4) | 20 | Max.size of memory heap (in kilobytes) | ||
| maxHeapSizeSet | 20 | Set max. size of heap | ||
| 20 | Max. size of archive input buffer in MB | |||
| maxCacheSet | 20 | Set max. size of archive input buffer (default 32 MB) | ||
| maxCorrCacheGet | 20 | Max. size of correction value input buffer in MB | ||
| maxCorrCacheSet | 20 | Set max. size of correction value input buffer | ||
| intervals | 1 | |||
| aliveInterval | 20 | Cycle time for time stamp (in seconds) | ||
| saveCycle | 20 | Cycle time for online backup of the current archive record (in seconds) | ||
| setMgmt | 1 | |||
| fileSwitch | 1 | |||
| switchTimeGet4) | 41 | Time of archive record change, reference to _TimedFunc is managed by the archive itself | ||
| switchTimeSet | 41 | Set time | ||
| switchRoundOffGet | 20 | Period in seconds that indicates when the values are actually written into the new archive. This means that the new archive record is created at the time switchTime but is activated switchRoundOff seconds later. The "real"switch time therefore is switchTime + switchRoundOff - | ||
| switchRoundOffSet | 20 | Set the transition period | ||
| overflowCountGet4) | 20 | min. number of empty value tuples per DPE in the compressed archive record, in the level 1. | ||
| overflowCountSet | 20 | Set min. number of empty value tuples | ||
| overflowPctGet4) | 21 | Percentage of empty value tuples in the compressed archive record in level 1 | ||
| overflowPctSet | 21 | Set the percentage of empty value tuples | ||
| overlapPeriodGet 4) | 20 | Length of time indicating how long both archive records remain open when the archive record is changed (in seconds) | ||
| overlapPeriodSet | 20 | Set time | ||
| expKeepPeriodGet4) | 20 | Hysteresis for expanded non-updated archive record (in seconds) | ||
| expKeepPeriodSet | 20 | Set hysteresis | ||
| fileCompression | 1 | |||
| packCountGet 4) | 20 | Number archive records in compression level 1 | ||
| packCountSet | 20 | Set the number of archive record | ||
| packTimeGet 4) | 41 | Time of compression, at level 2, reference to _TimedFunc is managed by the archive itself | ||
| packTimeSet | 41 | Set time | ||
| autoStartAfterSwitchGet | 23 | Starts automatically after a file change. If the value is TRUE the archives are compressed automatically. | ||
| autoStartAfterSwitchSet | 23 | Starts automatically after a set file change. If the value is TRUE the archives are compressed automatically. | ||
| fileMerge 4) | 1 | |||
| mergeCountGet 4) | 20 | Number of archive records used to define the level for overflow. This means that if this value of archive records is exceeded, the values are merged. | ||
| mergeCountSet | 20 | Set number of archive records | ||
| mergeTimeGet4) | 41 | Time for merge reference to _TimedFunc. It is managed by the archive itself | ||
| mergeTimeSet | 41 | Set time | ||
| fileDeletion | 1 | |||
| keepCountGet4) | 20 | Max. number of saved archive records, 0 = use only interval | ||
| keepCountSet | 20 | Set maximum number of saved archive records | ||
| removeTimeGet4) | 41 | Time of deletion. Reference to _TimedFunc. It is managed by the archive. The time to be saved is stored in the interval part of this DPE (in seconds) | ||
| removeTimeSet | 41 | Set time | ||
| files2) | 1 | transferred from archive at start and in case of changes | ||
| fileName | 9 | Array of all archive files (without path ), youngest first (0 = current) | ||
| state | 5 | Statuses of the single archive files (0 = online -1 = deleted) | ||
| compressionState | 5 | Compression states | ||
| startTime | 10 | Archive record starting times | ||
| endTime | 10 | Archive record end times | ||
| statistics | 1 | Values are set only after setting the index | ||
| index | 20 | Index in files.filename, which archive record should be queried (value can be written) | ||
| dpElementCount2) | 20 | Number of DPEs in the archive record | ||
| dpElements2) | 29 | All the DPEs of this archive record | ||
| dpValues2) | 5 | the number of value entries for each DPE | ||
| size2) | 20 | size of archive record | ||
Remarks:
-
can only be set when an archive is created, otherwise read-only
-
read-only
-
active beginning from the next archive start, read-only
-
active beginning from the next archive record change, read-only
Archive record change/compression/deletion
Archive record changes can be triggered automated (via archive parameter, see previous chapter) and /or manually. The _ValueArchive_X.action.fileSwitch.start DPE of the respective archive DP is used for the last method. The current status of the archive record change is displayed via _ValueArchive_X.action.fileSwitch.progress DPE. The same applies for the processes compression and deletion of archive records. If the processes mentioned above are initiated manually, the oldest archive record(s) are always affected (exception: in case of control via the configuration interface [..progress=2] the archive records affected can also be specified directly, see chapter 5). When deleting archive records, in addition to specifying the number of archive records to be retained, it is also possible to specify a time period (in the interval part of the removeTime element). A value of 0 (as the interval or count) means that this type of specification should not be used. If both parameters are specified, first the interval parameter and then the count parameter is taken into account. The methods described in this chapter are all executed by the archive itself or can be manually initiated with the .start dp element.
An archive is changed when the maximum number of values is reached. This can be prevented by setting maxFillPct to -1. In this case also values can be lost. errDp-DPE shows which DPE triggered the filling of an archive record. In this case, the errBit is also set to enable an alert handling. For compression, backup and deletion (in each case) 9999 archive sets can be configured.
The parameter switchRoundOff determines how many seconds after the change of an archive record, the write processes are actually changed to the other archive. Thus, it is possible to assign identical names to archives as well as to synchronize the times of archive record changes, in case of redundant systems.
Note that you set a proper value for the element setMgmt.fileSwitch.switchRoundOff of the internal ValueArchive DP (default is 30 seconds).
If a manual or time triggered archive set change takes place and the previous archive gets full during the period between the switch and "switchRoundOff" seconds, no additional archive set is created (normally an archive set would be created). In this case data may get lost. Also note that manual or time triggered archive set switches may not take place later than specified via the switchRoundOff setting. Later switches may cause different archive set names on redundant systems.
Time-triggered archive set switches during the file synchronization are now suppressed. Switches that should be executed during the backup, are executed after the backup (with original time stamping!).
Note that if archive sets are changed time triggered on an active system after the redu file synchronization and before the corresponding archive has started on the passive system, different archive set file names are created.
Table: DPE action of the DP _ValueArchive._ValueArchive
| Element | Element | Int | Purpose | ||
|---|---|---|---|---|---|
| action | 1 | ||||
| fileList | 9 | Contains the names of archive records to be compressed | |||
| fileSwitch | 1 | ||||
| start | 23 | Activation of archive record switch | |||
| progress | 23 | 1 = in progress, 0 = finished | |||
| fileCompression | 1 | ||||
| start | 23 | Activation of compression | |||
| progress | 23 | 1 = in progress, 0 = finished, 2 = Export fileList | |||
| targetCompression (not in use) | 20 | required compression level (0,1,2), only possible with fileList | |||
| fileMerge | 1 | ||||
| start | 23 | Activation of merge process: | |||
| progress | 23 | 1 = in progress, 0 = finished, 2 = Export fileList | |||
| fileDeletion | 1 | ||||
| start | 23 | Activation of deletion | |||
| progress | 23 | 1 = in progress, 0 = finished, 2 = Export fileList | |||
| fileBackup | 1 | ||||
| startTimeGet | 41 | Time of backup reference to _TimedFunc, is managed by the conf. interface | |||
| startTimeSet | 41 | Set time | |||
| saveCountGet | 20 | Number of archive records that are not to be backed up (beginning with the youngest one), 0 = all completed archive records are backed up | |||
| saveCountSet | 20 | Set number | |||
| start | 23 | Activation of the backup, see chapter 5) | |||
| progress | 23 | 1 = in progress, 0 = finished, 2 = Export fileList, 3 = Files ready for backup, 4 = Backup complete | |||
| fileRestore | 1 | ||||
| start | 23 | Activation of Restore (see chapter 5) | |||
| progress | 23 | 1 = in progress, 0 = finished, 2 = Export fileList | |||
The operation fileSwitch can only be activated by the archive or manually by setting the start element to 1. The operations fileCompression, fileMerge and fileDeletion can be activated by the archive itself, by setting the start element to 1 manually and via the configuration interface (see Data archiving). The operation fileRestore can only be activated via the configuration interface.
Backup/Swap-out/Deletion
The methods described here work with a cooperation of the configuration interface and the history level. The history level must check whether the process can be executed currently. If not, an error code is set and returned instead of a file list.
When the archive is started, the elements action.*.progress is set to 0 by the History DB (exception: fileBackup.progress, so that a started backup can be continued). When the configuration interface is started, all the statuses (..progress) are reset. If the status was > 0, the operation is restarted (repeat).
Online backup
In contrast to swap-out, online backup is used to back up all files of an archive. The backup is not executed by the archive itself. The archive only has to close all the files for a shortest possible time. An external triggered backup that is not part of the archive process or the history level can then back up these files. The system differentiates between the current archive record and the closed archive records. This means that these may be requested separately. For this reason the time period during which the current archive record may not be used should be minimized. The file names of the currently closed files are displayed in action.fileList.
An online backup can only be reimported when an archive is stopped. This process should also be initiated externally and is not part of the archive or history level.
Table: DPE onlineBackup of DP _ValueArchive
| Element | Element | Int | Purpose | |
|---|---|---|---|---|
| action | 1 | 1 | ||
| onlineBackup | 1 | |||
| startTime | 41 |
for automatic N/A reference to _TimedFunc., determines the length of the timeout of the archive: 0 = with _Timed Func , MAXTIME = no timeout, otherwise = time period = timeout |
||
| endTime | 41 |
for automatic N/A reference to _TimedFunc. Determines the length of the timeout of the archive: 0 = calculation with _Timed Func , MAXTIME = no timeout, otherwise = time period = timeout |
||
| startCurrent | 23 | Activates backup mode (current archive record) | ||
| progressCurrent | 23 |
Message from archive, 1 = files closed, file names saved in fileList , 0 = Online backup completed |
||
| startClosed | 23 | Activates backup mode (closed archive record) | ||
| progressClosed | 23 |
Message from archive, 1 = files closed, file names saved in fileList , 0 = Online backup completed |
||
Deletion of archive records
Deletion and/or swap-out processes are required for constant archive sizes. These always regard whole archive records (i.e. all DPEs of an archive for a particular time period). The file names of the files to be deleted must be transferred as parameters to action.fileList. The deletion can be triggered manually or time controlled and is activated and controlled via the configuration interface.
Redundancy synchronization: At start-up, the archive checks whether the transferred files list matches the contents of the archiving directory (state = online). Superfluous files in the file system are deleted.
Table: DPE lifeList of _ValueArchive
| Element | Element | Int | Purpose |
|---|---|---|---|
| action | 1 | 1 | |
| lifeList | 91 | List of file names in question |
Backup/swap-in closed archive records
The backup of archive records is similar to the deletion process. Each archive is started for a period of time. The configuration interface notifies the archive which archive records are to backed up and the archive locks them for write access during backup. The script archiv_client.ctl executes the actual swap-out. (for new projects the script is available in the script list pvss_scripts.lst of the CTRL manager,by default)
The element action.fileBackup.startTime can be used for cyclic activation.
An assignment of the archive record file name - backup medium (+ time period) can be stored by the backup script in the structures action.media and archive. This can be done to facilitate the swap-in process for the user. The swap-out process can be triggered manually or time-driven.
Swapping in is carried out in reverse.
Table: The DPE action.media of _ValueArchive
| Element | Element | Int | Purpose |
|---|---|---|---|
| action.media | 1 | ||
| change | 24 |
for script: if edge is 0 >1 exports new file list (in the external script), 1 > 0 if mediumFileList is updated |
|
| fileList | 9 |
Files on the current backup medium. device string is the device name of DAT, CD, MO under Linux, otherwise it is the path for CD, MO under Windows |
|
| progress | 21 | Device access numbers tape drive) | |
| redundancy device | 25 | Path name! DB for redundancy on external computer, if empty -> DO NOT copy!! | |
| hostName | 25 | The name of the computer where the backup is saved | |
| backupDevice | 25 | The device/ path where the backup medium is located is specified here |
Table: DPE archive of _ValueArchive
| Element | Element | Int | Purpose |
|---|---|---|---|
| archive | 1 | all the stored information, written by the archive script | |
| archiveFilenames | 9 | all the files names swapped out by the system (on an arbitrary medium; each archive only on one medium!!) | |
| archiveCompressState | 5 | compression mode [0,1,2,...] for each swapped-out data record | |
| archiveStartTime | 10 | readonly (Info copy of History DB). This is the start time for each data record | |
| archiveEndTime | 10 | readonly. This is the end time for each data record (necessary in case of any gaps !!) | |
| archiveUser | 9 | User who swapped out the data (or the system) | |
| archiveDate | 10 | Date and time of the swap-out |
When the archive is started, it checks the current state of the action.fileBackup.progress and continues an already started backup process.
Compression/Decompression of archive records
The archive records can be compressed either via the file system function under Windows (see compressValueArchive) or by configuring the compression of archives (beginning from the version 2.12.1) (see History DB - Compression of archives).
When the compression is called automatically, all compression levels are executed. 2 file lists are built one after the other:
For level 1:
All archive records located on the disk, except the current archive record, and the records with compression level smaller than 1.
For level 2:
All archive records with compression level 1 (i.e. the current archive record is not included), reduced by the number of archive records that should remain in the level 1 (DPE "packCount").
Both file listings have to be summated. After summating of the file lists, the list is abbreviated via the config entry maxNumberOfFilesToCompress (Default = 5, i.e. 5 archive records per archive are compressed each time during the automatic compression). The compression is initiated for each entry in the file list:
-
If the current level == 0 the compression in level 1
-
If the current level == 1 the compression in level 2
This is valid for the setting "automatic compression" (DPE "autoStartAfterSwitch") after an archive record change and for the setting "delay time" (DPE "expKeepPeriod"). The compression of archives can also be triggered manually (see panel History data base activity).
The following table gives a review of the most important internal data points and of the value range that can be configured in connection with the History DB compression:
| Data point element | Type | Range | Description |
|---|---|---|---|
|
ValueArchive_N.setMgmt.fileCompression. autoStartAfterSwitch |
bool | TRUE, FALSE | Activates the automatic compression after an archive record change. |
|
ValueArchive_N.setMgmt.fileSwitch. expKeepPeriod |
unsigned | 30-7200 | Delay time between archive record change and start of the compression. |
|
ValueArchive_N.setMgmt.fileCompression. packCount |
unsigned | 0-99 | Number of archive records, which should remain in compression level 1. |
|
ValueArchive_N.setMgmt.fileSwitch. overflowPct |
int | 0-100 | Percentage for the determination of free value entries per DPE. The value is only used for compression in level 1. |
|
ValueArchive_N.setMgmt.fileSwitch. overflowCount |
unsigned | 0-nnnn | A fixed value to be used as the amount of free value entries per DPE. Value is only used for the compression in level 1. The range is based on the configured number of value entries per DPE. |
