DBMS

A database is a collection of occurrence of multiple record types containing the relationship between records, data aggregate and data items. A database may be defined as
      A database is a collection of interrelated data store together without harmful and unnecessary redundancy (duplicate data) to serve multiple applications
      The data is stored so that they are independent of programs, which use the data. A common and control approach is used in adding the new data, modifying and retrieving existing data or deletion of data within the database A running database has function in a corporation, factory, government department and other organization. Database is used for searching the data to answer some queries. A database may be design for batch processing, real time processing or on line processing.

DATABASE SYSTEM
      Database System is an integrated collection of related files along with the detail about their definition, interpretation, manipulation and maintenance. It is a system, which satisfied the data need for various applications in an organization without unnecessary redundancy. A database system is based on the data. Also a database system can be run or executed by using software called DBMS (Database Management System). A database system controls the data from unauthorized access.

Foundation Data Concept:
A hierarchy of several levels of data has been devised that differentiates between different groupings, or elements, of data. Data are logically organized into:

Character
      It is the most basic logical data element. It consists of a single alphabetic, numeric, or other symbol.

Field
      It consists of a grouping of characters. A data field represents an attribute (a characteristic or quality) of some entity (object, person, place, or event).

Record
      The related fields of data are grouped to form a record. Thus, a record represents a collection of attributes that describe an entity. Fixed-length records contain, a fixed number of fixed-length data fields. Variable-length records contain a variable number of fields and field lengths.

File
      A group of related records is known as a data file, or table. Files are frequently classified by the application for which they ar primarily used, such as a payroll file or an inventory file, or the type of data they contain, such as a document file or a graphical image file. Files are also classified by their permanence, for example, a master file versus a transaction file. A transaction file would contain records of all transactions occurring during a period, whereas a master file contains all the permanent records. A history file is an obsolete transaction or master file retained for backup purposes or for long-term historical storage called archival storage.

Database

      It is an integrated collection of logically related records or objects. A database consolidates records previously stored in separate files into a common pool of data records that provides data for many applications. The data stored in a database is independent of the application programs using it and o the ‘type of secondary storage devices on which it is stored.

ETL Testing Online Training Course Content

We need to aware of below topics

Oracle Basics & Concepts
1. DBMS
2. RDBMS
3. DBMS vs RDBMS
4. Why Database Required
5. Different types of databases and difference
6. ASCII vs UNICODE
7. PL/SQL Basics
8. Oracle Architecture
9. Diff B/W Database & Files
10. OLTP
11. OLAP, ROLAP, MOLAP
12. METADATA
13. DDL,DML,DCL
14. BASIC ADMIN ACTIVITIES
15. DATATYPES
16. TABLES
17. SQL ,SUB QUERIES,CORELATED SUB QUERY
18. INNER QUERY,OUTER QUERY
19. FUNCTIONS AND TYPES AND IMPORTANCE
20. JOINS AND DIFFERENT TYPES
21. VIEWS n MATERIALIZED VIEWS
22. INDEX
23. CONSTRAINTS
24. REFERENTIAL INTEGRITY
25. PARTITIONING
26. PERFORMANCE TUNING
27. DIFFERENT TYPES OF TECHNIQES
28. DATABASE SCHEMA

DWH Concepts
1. WHAT IS DWH ?WHY WE REQUIRE THAT ?
2. DWH ARCHITECTURE
3. DATA MART and TYPES
4. DM vs DWH
5. DATA CLEANZING
6. DATA SCRUBING
7. DATA MASKING
8. NORMALIZATION
9. ODS
10. STG AREA
11. DSS

ETL Testing Course Contents

12. Diff B/w OLTP vs ODS,OLAP vs DSS
13. DIMENTION MODELING
14. DIMENSIONS
15. FACTS
16. AGGREGATES
17. DWH SCHEMA designing
18. STAR SCHEMA,SNOWFLAKE SCHEMA,GALAXY SCHEMA,FCS
19. SLOWLY CHANGING DIMENSIONS
20. SCD TYPE1,TYPE2,TYPE3
21. INITIAL LOAD
22. INCREMENTAL LOAD
23. FULL LOAD
24. CDC- change data capture
25. FAQ’S

ETL Testing Concepts
1. Introduction of ETL-Extract,Transform,Load
2. ETL TOOLS and Diff Types of ETL Tools
3. ETL ARCHITECTURE
4. ETL TESTING AND WHY WE REQUIRE
5. DIFFERENT ETL TOOLS ARCHITECTURES
6. SDLC and Methods/Models
7. STLC and Methods/Models
8. SDLC vs STLC
9. Reverse Engineering
10. QC(Quality Center and BugZilla)
11. Roles and Responsibilities
12. Minus,Duplicate,Count,Intersection,etc…
13. Detect Defects
14. Defects Logging and Reporting
15. How to prepare Queries very quickly with the help of mapping
16. Performance Tuning and Performance Testing,Report Testing,UI Testing
17. Quality and different standards that tester should follow,Why?
18. Testplan Preparation
19. Testcases Preparation
20. Preparation of Test data
21. Process Of ETL Testing

Testing Concepts
1. Whitebox Testing
2. Blackbox Testing
3. Gray Box Testing
4. Regression Testing
5. Smoke Testing vs Sanity Testing
6. User Testing
7. Unit testing
8. Intigration testing
9. Module testing
10. System testing
11. UAT

ETL Tool and Testing

1. Data Extract
2. Data Transform
3. Data Load
4. Import Source
5. Import Target
6. Mappings,Maplets
7. Workflows,Worklets
8. Transformations,Functionalities,Rules and Techniques
9. Import and Export
10. Coping and Rules
11. Queries Preparation based on Transformations
12. Importance of ETL Testing
13. Creating of Mappings,Sessions,Workflows
14. Running of Mappings,Sessions,Workflows
15. Analyzing of Mappings,Sessions,Workflows
16. Tasks and Types

Practice:

Testing scenarios, creation of test cases and scripts
Test case execution and defect tracking and reporting
Preparation of Test data
Practice ETL Testing with Real Time Scenarios and FAQ’s

Check the below link

The best Online Training for ETL Testing is as follows

 Call:  Sandeep Manem
          +91-8237320101
          +91-9885320101
Trained around 100+ students and placed around 90+ in Top MNC's with 90% success rate.

ETL Testing Challenges

ETL testing is quite different from conventional testing. There are many challenges we faced while performing data warehouse testing. Here is the list of few ETL testing challenges I experienced on my project:
- Incompatible and duplicate data.
- Loss of data during ETL process.
- Unavailability of inclusive test bed.
- Testers have no privileges to execute ETL jobs by their own.
- Volume and complexity of data is very huge.
- Fault in business process and procedures.
- Trouble acquiring and building test data.
- Missing business flow information.

Data is important for businesses to make the critical business decisions. ETL testing plays a significant role validating and ensuring that the business information is exact, consistent and reliable. Also, it minimizes hazard of data loss in production.

Hope these tips will help ensure your ETL process is accurate and the data warehouse build by this is a competitive advantage for your business.

Difference between Database and Data Warehouse Testing

There is a popular misunderstanding that database testing and data warehouse is similar while the fact is that both hold different direction in testing.

•  Database testing is done using smaller scale of data normally with OLTP (Online transaction processing) type of databases while data warehouse testing is done with large volume with data involving OLAP (online analytical processing) databases.
•  In database testing normally data is consistently injected from uniform sources while in data warehouse testing most of the data comes from different kind of data sources which are sequentially inconsistent.
• We generally perform only CRUD (Create, read, update and delete) operation in database testing while in data warehouse testing we use read-only (Select) operation.
• Normalized databases are used in DB testing while demoralized DB is used in data warehouse testing.

There are number of universal verifications that have to be carried out for any kind of data warehouse testing. Below is the list of objects that are treated as essential for validation in ETL testing:
- Verify that data transformation from source to destination works as expected
- Verify that expected data is added in target system
- Verify that all DB fields and field data is loaded without any truncation
- Verify data checksum for record count match
- Verify that for rejected data proper error logs are generated with all details
- Verify NULL value fields
- Verify that duplicate data is not loaded
- Verify data integrity

ETL Testing Process

ETL Testing Process:

Similar to any other testing that lies under Independent Verification and Validation, ETL also go through the same phase.

• Business and requirement understanding
• Validating
• Test Estimation
• Test planning based on the inputs from test estimation and business requirement
• Designing test cases and test scenarios from all the available inputs
• Once all the test cases are ready and are approved, testing team proceed to perform pre-execution check and test data preparation for testing
• Lastly execution is performed till exit criteria are met
• Upon successful completion summary report is prepared and closure process is done.

It is necessary to define test strategy which should be mutually accepted by stakeholders before starting actual testing. A well defined test strategy will make sure that correct approach has been followed meeting the testing aspiration. ETL testing might require writing SQL statements extensively by testing team or may be tailoring the SQL provided by development team. In any case testing team must be aware of the results they are trying to get using those SQL statements.

ETL Testing Techniques

ETL Testing Techniques:

1) Verify that data is transformed correctly according to various business requirements and rules.
2) Make sure that all projected data is loaded into the data warehouse without any data loss and truncation.
3) Make sure that ETL application appropriately rejects, replaces with default values and reports invalid data.
4) Make sure that data is loaded in data warehouse within prescribed and expected time frames to confirm improved performance and scalability.

Apart from these 4 main ETL testing methods other testing methods like integration testing and user acceptance testing is also carried out to make sure everything is smooth and reliable.

why ETL Testing require for Data Warehouse

Why do organizations need Data Warehouse?
Organizations with organized IT practices are looking forward to create a next level of technology transformation. They are now trying to make themselves much more operational with easy-to-interoperate data. Having said that data is most important part of any organization, it may be everyday data or historical data. Data is backbone of any report and reports are the baseline on which all the vital management decisions are taken.

Most of the companies are taking a step forward for constructing their data warehouse to store and monitor real time data as well as historical data. Crafting an efficient data warehouse is not an easy job. Many organizations have distributed departments with different applications running on distributed technology. ETL tool is employed in order to make a flawless integration between different data sources from different departments. ETL tool will work as an integrator, extracting data from different sources; transforming it in preferred format based on the business transformation rules and loading it in cohesive DB known are Data Warehouse.

Well planned, well defined and effective testing scope guarantees smooth conversion of the project to the production. A business gains the real buoyancy once the ETL processes are verified and validated by independent group of experts to make sure that data warehouse is concrete and robust.

ETL or Data warehouse testing is categorized into four different engagements irrespective of technology or ETL tools used:

• New Data Warehouse Testing – New DW is built and verified from scratch. Data input is taken from customer requirements and different data sources and new data warehouse is build and verified with the help of ETL tools.
• Migration Testing – In this type of project customer will have an existing DW and ETL performing the job but they are looking to bag new tool in order to improve efficiency.
• Change Request – In this type of project new data is added from different sources to an existing DW. Also, there might be a condition where customer needs to change their existing business rule or they might integrate the new rule.
• Report Testing – Report are the end result of any Data Warehouse and the basic propose for which DW is build. Report must be tested by validating layout, data in the report and calculation.