Data Dictionary

The Data Dictionary contains data on the following:

• All fields defined in all input forms, and some internal fields generated automatically (e.g. "foreign keys").

  • Note that a given field (identified by name) may be used in more than one table.

  • Some field types define enumeration values (e.g. dropdown, list, group or table of radiobuttons). For fields of these types, the Data Dictionary contains the list of enumerations value and associated texts.

• All dynamically generated tables.

  • The root table (currently the root table is always "Patient") is stored in the Data Dictionary, too - but only for the purpose of defining relations.

• The relationship between tables and fields.

• The relationship between tables (1:1 or 1:N relations).

• Computations per table (see Computation Reference) - used for:

  • Texts and styling of commands in the patient record tree.

  • Security

  • (rudimentary) Workflow

In the main menu of the web application there is an item "Configuration" with a sub-item "Data Dictionary". Activating this sub-item displays a sub-menu (tabstrip) with the following commands:

• View (browse) the Data Dictionary.

  • The left pane displays the dynamic data structure as a tree (more or less as defined in The Structure File).

  • Each node in this tree corresponds to one table in the Data Dictionary. Clicking a node opens a page that shows information on the table in the right pane.

• Print the Data Dictionary.

  • All information in the Data Dictionary is displayed in one page to make it possible to print the contents (or export to some other system).

• Update the Data Dictionary (see below).

• Delete the Data Dictionary.

• Log

  • Each run of the Create/update Data Dictionary generates a set of log files, that are stored in separete folders having time stamps as folder names).

• Webservices

  • Displays all configuration (application) specific webservices - auto-generated as well as custom.

Changes to The Structure File, ALWAYS require updating the Data Dictionary - otherwise the performed changes will not have any effect.

Some, not all, changes to form templates will also require updating the Data Dictionary.

• Changes that adds or removes fields, or change field type, etc. will require updating the Data Dictionary - otherwise errors may occur when trying to save data entered in dynamic forms.

  Tip
  Edited form templates (with changes to fields) may be tested in CREATE MODE. Attempting to save data entered in such forms will generate errors.

• Changes to form templates that only change sequence of fields, location in form, layout, styling, rules, JavaScript, etc. will NOT require updating the Data Dictionary.

Updating the Data Dictionary involves:

• The system constructs a new Data Dictionary in memory, and compares it to the Data Dictionary stored in the database. The detected differences are converted to SQL DDL commands (CREATE TABLE, DROP TABLE, ALTER TABLE, etc.)

• Caution
  It is imperative, that you know what you are doing in this process. If you for example remove a form from the structure file, the corresponding database table will be DROPPED from the database, dicarding any data stored in that table (without the possibility for undoing)!

• To help the configurator check (and double check!) that data is not unintentionally dicarded, the command to update the Data Dictionary may be run in test mode. This mode displays debug output that shows which modifications will be done to the database without actually performing these changes. Updating the Data Dictionary should always be done in test mode first, and the output analysed. If this output shows the expected changes, then the update may be run again with test mode turned off. Naturally this is especially important when updating Data Dictionary in applications that run in production mode.

• Each run of the Create/Update Data Dictionary generates a set of log files (the files from each run in stored in a separte folder). These log files contain more information thatn the ouput that is automatically displayed after updating Data Dictionary. When in doubt, consult these log files (thery are accessible from the web user interface - the "Log" tab).

• If the update is not run in test mode, the generated DDL is executed (updating the database table structure), and the contents of the Data Dictionary (the metadata in the database) is updated to reflect the new updated data structure in the database.

In "normal" database design, a given field name may be used for different field types in different tables. E.g. a field X may be an integer in table Table1 and a text field in table Table2.

This is NOT the case in TOPICA. The TOPICA Data Dictionary contains a central list of fields (identified by field name only). If a given field name is used in several forms, the type of of the corresponding field will be the SAME in all the tables generated from the forms.

This is because TOPICA contains functionality to make field values "inherit" across forms / tables. I.e. the latest value of all fields with same name in all forms may be automatically set as the defult value for a field, when opening a form in "create" mode.

See also field names.

If the configurator / form designer uses a given field name in different forms, and does NOT make sure to use SAME field type in all forms, the system will be confused - because then there will be inconsistencies between how a field is defined in the form, and how the field is created in the database table.

When building/updation the data dictionary, some checks to catch this is performed. The amount of checking depends on the TOPICA version:

• TOPICA version 4.30 or earlier:

  • Field name used for different data types.

  • Field name used for enumration, and there is a conflict in the the Caption property.

  The warning messages are not so user friendly. It is possible to see in which forms the fields are redefined - but there is no list of all the conflicting field definitions per field name.

• TOPICA version 4.31 or later performs the same checks as described for 4.30 above - plus these:

  • Field name used for text fields with different sizes.

  • Field name used for enumeration, and there is a conflict in the the EnumName property.

  The warning messages are more user friendly: After all forms / fields have been processed, a combined list of all the conflicting field definitions are listed per field name. This list and a SQL-script to check max. length of contents per field name are saved in the log files.

To see this warnings in action, you may play around with the demo application DemoDataDictionary. The forms in this application contain som deliberate design errors:

• Some field names are used for fields with different data types.

• Some field names are used for text fields with different sizes.

• Some field names are used for enumerations (dropdowns) with conflicting values for Caption or EnumName.

In case of text fields where sizes (specified by property Width on the TextLine elements) differ, the system automatically adjust the sizes (used for generating fields in the databse), so that the LARGEST value is used.

Example (from the DemoDataDictionary application):

• Forms:

  • Form defined in Form1.xml contains a field "TextLine1" with Width=10.

  • Form defined in Form2.xml contains a field "TextLine1" with Width=20.

  • Form defined in Form3.xml contains a field "TextLine1" with Width=30.

• As specified in the structure file (Structure.xml):

  • Form "Form1" generates table "Form1".

  • Form "Form2" generates table "Form2".

  • Form "Form3" generates table "Form3".

• All tables "Form1", "Form2" and "Form3" will contain a field "TextLine1" with size 30.

• The confusing part is, that when looking at the form definition "Form1", one would expect the field "TextLine1" in table "Form1" to be of size 10 - not 30.

As a special case, if a field name is used for a TextLine (finite size, e.g. 10) in one form, anden for at TextArea (infinite size), the field for the TextLine form element will generate a database field of type VARCHAR(MAX), not VARCHAR(10) as it would normally be, had the field not been "redefined" in an other form.

Build/update Data Dictionary for the DemoDataDictionary application to see the warnings.

Note that all warnings are stored in logfiles for the run, that detected the warnings. Warnings cannot be listed by running a report, as the necessary information is not stored in the database. The only way to get these warnings is to run the "Update Data Dictionary" command (run in test mode to avoid actual updating).

The set of log files also contains a SQL-script, that computes the max. length of the text field contents (having inconsistent sizes) per field name. When a text field has been defined with too large value for Width, use this script to check whether it would be acceptable to make the field size smaller.