AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |
Back to Blog
![]() ![]() To facilitate this post-recovery reconstruction of records, each event is tagged with an ID upon its arrival from the event generator. In the proposed design, such loss will be recovered by retrieving the events from the disk and reconstructing the lost records. ![]() Also lost will be any records generated but not yet delivered to the record collector. In the event of a transaction system failure, all events stored in processor memory will be lost. All events remain on disk until their corresponding records have been generated, delivered to, and confirmed by the record collector. The present invention is such a system.Īll events received by the transaction system are stored both on a disk and in memory: on disk to prevent memory loss due to failure, and in memory to facilitate processing with minimal latency. Thus, transaction processing under normal conditions can be increased by accepting a somewhat higher processor overhead during recovery after a failure. Since failure rates are generally low, a scheme which results in larger processing demands upon recovery after a failure is acceptable. A data integrity scheme which presents low overhead during normal processing is desirable. However, when they occur, large amounts of data may be lost during each failure. In a high throughput system, failure rates may be low. To the contrary, these demands are overhead processing costs. Data integrity schemes present additional demands on processor utilization which reduce the capacity of the system. In high throughput systems, the processor must be utilized mainly for transaction processing. This is likely because for every transaction that has received a new event since the last disk update, all events belonging to the transaction get written to disk to maintain the structural relationship. For example, the same event may be written to disk multiple times. However, the storing of raw events in conjunction with structural information consumes processing resources, and the amount of processor time required by this method can become excessive as transaction volume increases. Such disk-stored structural information is useful to simplify the reformatting of records upon system recovery after a failure. ![]() Thus, in addition to storing the raw events, the structure of the events in relation to the transactions gets stored as well. However, in this technique, events are stored on disk only after they have been assembled and correlated to their transactions. A second technique improves upon the first by saving events. This method is used to reduce processing time and to conserve disk space since a record is typically much smaller than the total size of all the events related to the record.Īs technology has progressed, inexpensive disk storage has become more readily available. However, in this strategy, if a failure occurs, events which have not yet been formatted into records are lost. The first is to store records after they have been formatted. Traditionally, two data storage techniques have been used to ensure data integrity. Thus, there are three primary stages in the flow of data through the transaction system: event receiving, record formatting, and record forwarding. This record is then forwarded to another system, a record collector, for further processing. When the last event is received, a record for the transaction is generated by the transaction system. In one type of transaction system, during the lifetime of each transaction, a series of several events containing raw data related to the transaction is produced by an event generator and received by the transaction system. However, the additional processing required to attain this goal can have significant impacts on the other performance measures. As the transaction volume increases, it is critical that a high data integrity objective be maintained since even a very small fraction of data loss can result in the absolute loss of a relatively large number of transactions. During processing, data stored in memory of the transaction system will be lost if there is a failure that brings the system down, e.g., a processor failure.Ī transaction system has four primary performance measures: capacity, latency, data integrity, and availability. The system may also transmit data for further processing. Generally, a transactional computer system receives, processes and stores data. More particularly, the invention relates to a failure rate independent system for achieving zero defect data integrity in a high capacity transaction system. This invention relates generally to transactional computer systems for the processing of data. ![]()
0 Comments
Read More
Leave a Reply. |