The evolution of data management technology : from traditional file to data warehouse

Traditional File Processing Images

Traditional File Processing

Each business application was designed to use one or more specialized data files containing only specific types of data records. One of the ways to store,manipulate and retrieve large files of data. Traditional file based system is basically a file based system, in which we manually or through computer handle the database such as updating,insertion,deletion adding new files to database etc.

Problems with Traditional File Processing

Data Redundancy,Lack of Data Integration,Data Dependence and Lack of data integrity or standardization.

Data Management Approach

Consolidates data into databases that can be accessed by differently programs. It use a database management system (DBMS). DBMS-set of computer programs that controls the creation, maintenance, and the use of the database with computer as a platform or of an organization and its end users. A DBMS is a system software package that helps the use of integrated collection of data records and files known as databases. It allows different user application programs to easily access the same database. DBMS may use any of a variety of database models, such as the network model and rational model.

In large systems, a DBMS allows users and other software to store and retrieve data in a structured way. Instead of having to write computer programs to extract information, user can ask simple questions in a query language. Thus, many DBMS packages provide Fourth-generation programming language (4GLs) and other application development features. It helps to specify the logical organization for a database and access and use the information within a database. It provides facilities for controlling data access, enforcing data integrity, managing concurrency controlled, restoring database.

Database Management Approach Images

Data Warehouses

Repository of an organization's electronically stored data. Data warehouses are designed to facilitate reporting and analysis. Data warehouse focuses on data storage.

However, the means to retrieve and analyze data, to extract,transform and load data, and to manage the data dictionary are also considered essential components of a data warehousing system. Many references to data warehousing use this broader context. Thus, an expanded definition for data warehousing includes business intelligence tools, tools to extract,transform and load data into the repository, and tools to manage and retrieve metadata.

Data Warehouse Images

Comparison between General Application Software and Function-specific Application Software

General purpose application software performs common information processing jobs for end users but not in support of a specific purpose. Because they are significantly increases the productivity of end users, they are sometimes known as productivity packages.

Spreadsheet program is a general purpose application. It does spreadsheets, but those could be spreadsheets that you use to balance your checkbook, or manage your fantasy football league.

Database software is a database contains a list of information items that are similar in format and/or nature. An example is a phone book that lists a name, address, and phone number for each entry. Once stored in a database, information can be retrieved in several ways, using reports and queries. For example, all the names listed for a given area code could be printed out and used for a commercial mailing to that area.

Word-processing software previously done on typewriters with considerable effort can now be easily completed with word-processing software. Writing tasks such as keying in reports, letters, and tables, as well as merging documents, can be performed easily.

Function-specific application software is more limited in what it will do, but it usually goes that one thing much better than a general purpose program. You can use general purpose software for more purposes but limited capabilities for each purpose; but you can use function-specific software for few purposes but with greater capabilities for each purpose.

TurboTax (a tab preparation package) is a special purpose application. Sure, it adds and subtracts numbers like a spreadsheet, but you can't use it to track your fantasy football league. It does one thing - prepare tax returns.

Business application software supports the re engineering and automation of business processes with strategic e-business application like customer relationship management, enterprise resource planning, and supply chain management.
Other examples are software packages that Web-enable electronic commerce applications or apply to the functional areas of business like human resource management and accounting and finance.

Data mining, enterprise information portals, or knowledge management systems are software empowers managers and business professionals with decision support tools.

Evolution from mainfrace to microcomputer

Mainframe is the parent of all computers, this computer used mainly by large organizations for critical applications, typically bulk data processing such as census, industry or consumer statistics, and financial transaction processing. Therefore, mainframe can reduces management and administrative costs while providing greatly improved scalability and reliability. Mainframes can replaces hundreds of smaller because it often support thousands of simultaneous users who gains access through specialized terminals.

The term originated during the early years of computing and referred to the large mechanical assembly that held the central processor and input or output complex. It is only supported time sharing mode where users had no direct access to the computing service. Since the mid1960's, mainframe designs have included several subsidiary computers which manage the input and output devices, leaving the CPU free to deal only with high-speed memory. Mainframes commonly have hundreds to thousands of times as much data storage online, and can access it much faster. Mainframe like computing platform, it is dependent on its ability to scale, support mixed workloads, reduce labor costs, deliver uninterrupted service for critical business applications, and several other risk-adjusted cost factors. However, mainframe allow for "graceful degradation" and service while the system is running. Mainframes did not utilize programming languages that were user friendly. It is also huge in size, expansive and difficult to maintain.

IBM 704 mainframe

In 1940s, the first mainframe computers were developed then several manufacturers also produced mainframe computers.

In 1970s,terminals used for interacting with mainframe systems were gradually replaced by microcomputers. Consequently, demand plummeted and new mainframe installations were restricted mainly to financial services and government.

In the early 1990s, there was a consensus among industry analysts that the mainframe was a dying market as mainframe platforms were increasingly replaced by microcomputer networks. Counselors began to interface with microcomputers much more, it is prompt within the world of business.

As technology advanced the classifications mainframes have become blurred and modern microcomputers are as powerful as the mainframes of twenty years ago.

Microcomputer is a single user computer system based on an 8-bit microprocessor. These were used for small industrial, office and program development applications. A microcomputer contains a central processing unit (CPU) on a microchip (the microprocessor), a memory system (typically read-only memory and random access memory), a bus system and I/O ports, typically housed in a motherboard.

The advent of the microcomputer made computing time a much cheaper resource. Microcomputers also were increasing in computational power exponentially. The micro-computer answers our wishes for that intermediate device. It is offer both computing power and portability, thus brings it closer to the ability of a mainframe computer. Central system processing load is reduced compared with mainframe. The data models used by microcomputer database management system are simple, allowing users to design a database that matches their information needs exactly. Microcomputer data manipulation languages tend to be simple, so that users are able to perform the operations needed without extensive training. Hardware and software advances in the microcomputer field are so rapid that these systems are now capable of performance many of the functions of the mainframe database systems. End users recognize that the microcomputer provides a solution to their immediate computing. Microcomputer systems are relatively inexpensive and require less technical expertise than do mainframe system. So that start-up is easy.

Early 80s microcomputers


BBC Microcomputer RPI Calibration System


Microcomputer for Business


Business Computer Phase II


The world's long-lost first microcomputer

The first PC

Video Game Consoles




Desktop computer

Laptop