Advanced ETL Processor Data Validator guarantees to your business that every data value is correct and accurate. It checks the data against a set of user-defined rules and constraints and identifies any errors or inconsistencies that may be present. This helps organizations avoid problems such as incorrect data being loaded into their systems, or data being lost or corrupted during the ETL process
What is Data Validation
Data validation is the process of ensuring that data meets certain criteria or rules. It helps identify inaccuracies, inconsistencies, or anomalies within datasets. Here are some common ways to validate data:
Data Type Validation: Verifies that data is in the expected format and adheres to predefined data types. For example, ensuring that a field intended for numerical data only contains numbers.
Range and Boundary Validation: Checks if data falls within predetermined ranges or boundaries. It helps detect outliers or invalid values. For instance, validating that a temperature reading is within a specific range.
Format and Pattern Validation: Ensures that data conforms to a specified format or pattern. This is particularly useful when dealing with data such as email addresses, phone numbers, or postal codes.
Consistency Validation: Compares data across multiple sources or fields to identify discrepancies or conflicts. It helps maintain data integrity by ensuring consistency within datasets.
Referential Integrity Validation: Verifies that relationships between different data elements are maintained. For example, confirming that foreign key values in a database table exist in the referenced primary key column.
It does not matter which business you are in sooner or later you will discover that there is something wrong with the data and it has to be validated. Here when Advanced ETL Processor validator can help.
Records can also be rejected by the Server.
If there are several validation rules and one of them rejects the record and another discards it, the record will be discarded
To change Validator properties double click on it
If no data is present in the grid check the previous step execution log.
About input and output fields:
List of Output fields is always the same as Inputs.
List of Input fields is taken from the previous object
Create new tab
Snap to grid/show grid
Print Preview Mapping
Delete All objects
Delete All Links
Show Objects Panel
Zoom back to 100%
We recommend giving the Validator name, it makes it easier to identify problems later.
To start debugging validation press the Process Data button
. To test data edit it in the Data grid.
To Change Validation Rule properties double click on it (or right click and select properties)
Use <value> to include actual value into default value
To add a new Validation rule drag and drop it from the Validation rules panel
It is also possible to apply several validation rules to the Input field by joining them
Data is considered validated when all validation rules are succeeded.
If you have several validation rules and one of them rejects the record and another discards it, the record will be discarded