During the execution of a loader set, the target tables will be populated by inserting the extracted rows into each table. These rows will form a referentially correct and complete (as specified by the extraction rules) collection of subset data.
A Load Manager rule is a container which manages the execution of the subsidiary rules which perform the actual transfer of the data from the source schema into the target schema. Usually a single Load Manager rule is defined in a loader set and this rule is usually automatically generated when a loader set is created by the New Loader Set Wizard.
The implementation of the load operations as a manager rule was done to make the process simple and intuitive. Usually there will be many tables which will require rows in an Oracle schema and each load action requires the execution of a distinct SQL statement. Every operation must be recorded in the log and any errors must be detected and reported. If such actions were configured as individual rules in the loader set, then the large quantity of such rules could visually overwhelm the loader set - leaving it hard to maintain the other load rules. It is much more efficient to collect all of the actions together under one rule which acts as a manager. The Load Manager Rule form provides the ability to view, configure and manage the load actions and the Load Manager rule itself controls the execution of the configured load operations. Errors and statistics are handled by the Load Manager rule and presented in a manageable summary form.
It is possible to configure the Load Manager to ignore a certain number (or all) of the errors received while performing the load operations. This enables the Load Manager to continue operating even if errors occur. Be aware that ignoring the errors in the Load Manager just permits the Load Manager to complete as many actions as possible before returning an error and stopping the execution of the loader set. In other words, even if errors are ignored, any errors which occur during the load process will mark the Load Manager rule as having failed and the error state will be reported to the DataBee software for handling once the actions have completed.
Important Note: The Load Manager rule is only aware of the tables given to it by the Rule Controller. To refresh the list of tables known to the Load Manager rule, first refresh the tables in the Rule Controller using the Refresh Tables and Indexes button on Options tab of the edit Rule Controller form. This retrieves an up-to-date list from the target schema. Once the Rule Controller has been updated, use the Refresh Table List button on the Load Manager Rule form to refresh the list inside the Load Manager rule.
By default, the Load Manager rule assumes that the structure of the tables in the source and target schemas are identical. In other words it is assumed that the columns in the table have the same name and the same datatype. If this is not the case, the Load Manager rule contains a transformation engine which can manipulate the column information as necessary. The transformation engine modifies the data stream as it is read from the source table and inserted into the target table. It can remove data columns, add data items and perform function based modifications such as TO_STRING or TO_DATE on the existing values. The transformation engine help file contains detailed information on its configuration and use.
Once the Load Manager rule has begun to execute, the load of each table will proceed in parallel using the full number of threads specified on the Set Loader Run Statistics tab until all specified tables have been operated on.
Loader set rules always execute in the target schema and pull the data through a database link or via a direct schema-to-schema copy from the source table into the target table. At no time is the data transmitted to the PC running the Set Loader application. The Set Loader software simply initiates and controls the load process. The choice to use a database link or direct copy is set using the options on the Load Manager Rule form. If a database link is used, the link must run from the target schema into the source schema. If the direct copy mode is used, the target schema must be located in the same database as the source schema and the target schema must, at minimum, have SELECT access to the data in the tables in the source schema. Direct copy mode is much faster than loading via a database link - but has the drawback that the source and target schemas must be co-located in the same database.
Load Manager rules are usually configured using Rule Blocks to run after the Foreign Key Disable, Trigger Disable and Truncate Manager rules have completed. Once the Load Manager rule has finished, subsequent Foreign Key Enable and Trigger Enable rules are usually scheduled to execute. The disablement of the foreign keys ensures that errors relating to the referential correctness of the data are not generated while the load is in process. The Load Manager cannot ensure the continuous referential correctness of the data while the load operations are being performed. The Foreign Key Manager rule help file contains a lengthy discussion of this topic.
Load Manager rules are created by launching the New Load Manager rule form using the New Loader Rule button located on the bottom right of the main Set Designer form.
How to Create a New Load Manager rule