StatusQuo

Automatically refactoring database applications for performance

hidden
hidden

What is it?

StatusQuo is a new programming system for developing database applications using the Java language. It uses program analysis techniques to do two things:

  1. Identify parts of the program that operate on database result sets and convert them into SQL. For example, our system converts a nested loop on a Java ResultSet into equivalent SQL to allow the database pick the best join algorithm.
  2. Move blocks of code into the database as stored procedures, to reduce the number of network round trips and move code closer to data.

StatusQuo maintains the status quo, because it the programmer doesn't need to modify his or her code at all. Programmers write applications in Java as they always have, and StatusQuo optimizes the application during compile and execution time. There is no need to install any custom database or app server.


Specifically, StatusQuo aims to address the following issues that frequently arises when developing database applications:

Writing SQL

Developers often need to embed SQL statements in the application to interact with the database. Writing efficient SQL statements is hard. ORMs allow developers to express their persistent data operations without understanding SQL, but the SQL statements that are generated by the ORM backends are often inefficient because they lack whole program information.

Moving code

For the best performance, developers often push application logic into the database as stored procedures. Unfortunately, stored procedures are often a pain to write, debug, and maintain -- when was the last time that you wrote a non-trivial UDF?

Changing server loads

If the database is already heavily loaded, then moving computation to it will hurt performance. It is difficult for developers to anticipate server loads during initial development, and writing code to dynamically choose which portion of the application logic to push to the database is just a big hassle.

Security

App servers and database often have different security models. Ensuring that the security measures implemented in the database and app server are enforcing the same policy is not easy. How can you be sure that sensitive data fetched from the database are not leaked to some untrusted code in the application?


hidden
hidden

Design

StatusQuo is a system that aims to address the issues above without forcing developers to learn new languages or tools. The system is designed to automatically optimize, deploy, and adapt to changing server loads while maintaining security and transactional consistency of the original application.

The diagram below shows the design of StatusQuo. StatusQuo currently takes in source code written in Java that uses JDBC or Hibernate ORM and goes through the following steps to generate an efficient implementation:

Profiling

The system first compiles and deploys the application as-is and collects profiling data to capture information about the current workload.

Declarative code extraction

The source is passed to the extractor component, which tries to convert as many blocks of Java code into SQL as possible. This gives the system flexibility regarding how and where to implement the given functionality. For instance, if the database is under-utilized, the extracted code will be pushed to the database and executed as SQL. Otherwise, the extracted code can be converted back to Java and executed on the app server instead.

Code partitioning

The partitioner uses the workload data to split the source code into two programs: one to be executed on the database server as stored procedures, and another to be executed on the application server. The goal of partitioning is to reduce data transfers and the number of round trips between the two servers.

Dynamic tuning

After deployment, the runtime components on the servers monitor server loads and the performance of the partitioned program. Should the load exceeds the original level that the partitioned program was designed for, the runtime contacts the partitioner to generate a new partition for the current load, and the application is automatically re-deployed.


hidden
hidden

Publications

Here are some papers related to StatusQuo:


hidden
hidden

Technologies

We have experimented with TPC-C, TPC-W, and Hibernate applications for the individual components (see publications below for details). StatusQuo is made possible by two key technologies:

QBS

QBS (Query By Synthesis) is a tool that extracts relational specifications from imperative code fragments. It uses the Sketch synthesizer to find a provably correct transformation from Java into SQL.

Pyxis

Pyxis is a program partitioner for database applications. It takes in source code written in Java and figures out a way to split the source into two halves with the goal to minimize the number of roundtrips and data transferred between the two.


hidden
hidden

People

StatusQuo is a collaboration between the Database and CAP groups at MIT, and the APL group at Cornell University.

We gratefully acknowledge the support from NSF, NDSEG, and Intel for their support of this work.


hidden
hidden

Download

StatusQuo is currently under heavy development. We are currently preparing to release our project. If you are interested in trying out our system please send us an email and stay tuned!