David Maier
Next-Generation Data Stream Services
Friday, November 21, 2008, 10:30am
Sennott Square 5317
Refreshments and meet the speaker at 10:00am
Abstract: Data-stream systems now exist as advanced research prototypes and initial commercial offerings. However, these systems have limitations that restrict their performance and range of application, particularly as regards to dependence on physical stream properties. We believe out-of-order processing (OOP) techniques will be essential to support capabilities in the next generation of commercial data-stream management systems, such as distributed execution, data prioritization, traffic shaping, and interaction with transactional data sources. I will introduce our work on OOP architectures and order-agnostic operator implementations, including evaluations in both the NiagaST system at PSU and Gigascope at AT&T labs. In addition to expected benefits, such as reduced memory requirements and latency, we have seen improvements in bandwidth resulting from more flexibility in traffic shaping with the OOP approach. I will also discuss adaptive operators that can cope with dynamically changing data skew often seen in network applications. I will conclude by considering the connection of OOP capabilities with other next-generation stream-processing requirements.