No items found.

Announcing Data Engine from Periscope Data

Here at Periscope Data, we've had the privilege of working with and learning from more than 1,000 data teams. While there are many different and successful tech stacks for analytics, they all share common workflows and challenges around ingesting, modeling and reporting on data. Data Engine is designed to help data teams apply the right technology to the right workflows, eliminating challenges they face today.

Our customers use data lakes, often with Presto or Spark, to store and transform their largest and most raw data. From there, they usually use Redshift, Snowflake or another warehouse to transform and model the data backing their analytics. Some even move data from their warehouses to specialized databases optimized for certain kinds of analysis.

As data volumes scale and tooling becomes more complex, challenges begin to mount. It's hard to reliably move data from source to analysis-ready models. Another challenge is validating the quality along the way, making sure that each transform is running efficiently and that data doesn't get out of sync. It's hard to coordinate jobs between different warehouses while ensuring that the dependent analysis will be correct.

With Data Engine from Periscope Data, customers will be able to optimize the data pipeline for their unique workloads and data needs, all within a single platform. It gives data teams the power and flexibility to move data from anywhere to anywhere, apply business logic and orchestrate the entire data pipeline from ingest all the way through to visualization without having to learn and maintain a myriad of tools and technologies.

The most valuable thing that data teams can do is help their businesses make better decisions. Every moment spent debugging an ETL pipeline or figuring out why the data isn't right is a distraction from that goal. Data Engine automates a lot of the effort that doesn’t go toward business outcomes so the data team can focus what they do best, which is using the right tech at the right time for the right analytics problem.

With the initial rollout of Data Engine, Periscope Data customers will be able to choose whether to process their workloads on Snowflake or on Redshift, with more options being added in the coming months. As more data processing technologies are added to Data Engine, data teams will be able further optimize their performance and run even more efficiently. Now, instead of spending resources managing and optimizing their data technology stacks, data teams can focus on critical value-add tasks for their business.

For more information about Data Engine from Periscope Data, check out a video from our team explaining the project’s mission below. To get started leveraging the power of agile data warehousing, please contact your Periscope Data representative or sign up here to try it out for yourself.


Want to discuss this article? Join the Periscope Data Community!

Tom O'Neill
Periscope Data was started in Tom's apartment, where he built the first version of the product in a weekend. He leads Periscope Data's engineering efforts, and holds the coveted customer-facing bugfix record. (10 minutes!)