Skip to main content

Databricks makes bringing data into its ‘lakehouse’ easier

Databricks today announced that launch of its new Data Ingestion Network of partners and the launch of its Databricks Ingest service. The idea here is to make it easier for businesses to combine the best of data warehouses and data lakes into a single platform — a concept Databricks likes to call ‘lakehouse.’

At the core of the company’s lakehouse is Delta Lake, Databricks’ Linux Foundation-managed open-source project that brings a new storage layer to data lakes that helps users manage the lifecycle of their data and ensures data quality through schema enforcement, log records and more. Databricks users can now work with the first five partners in the Ingestion Network — Fivetran, Qlik, Infoworks, StreamSets, Syncsort — to automatically load their data into Delta Lake. To ingest data from these partners, Databricks customers don’t have to set up any triggers or schedules — instead, data automatically flows into Delta Lake.

“Until now, companies have been forced to split up their data into traditional structured data and big data, and use them separately for BI and ML use cases. This results in siloed data in data lakes and data warehouses, slow processing and partial results that are too delayed or too incomplete to be effectively utilized,” says Ali Ghodsi, co-founder and CEO of Databricks. “This is one of the many drivers behind the shift to a Lakehouse paradigm, which aspires to combine the reliability of data warehouses with the scale of data lakes to support every kind of use case. In order for this architecture to work well, it needs to be easy for every type of data to be pulled in. Databricks Ingest is an important step in making that possible.”

Databricks VP or Product Marketing Bharath Gowda also tells me that this will make it easier for businesses to perform analytics on their most recent data and hence be more responsive when new information comes in. He also noted that users will be able to better leverage their structured and unstructured data for building better machine learning models, as well as to perform more traditional analytics on all of their data instead of just a small slice that’s available in their data warehouse.

 



Udimi - Buy Solo Ads from TechCrunch https://ift.tt/38XI01I
via IFTTT

Comments

Popular posts from this blog

9 VCs in Madrid and Barcelona discuss the COVID-19 era and look to the future

Spain’s startup ecosystem has two main hubs: Madrid and Barcelona. Most observers place Barcelona first and Madrid second, but the gap appears to close every year. Barcelona has benefitted from attracting expats in search of sun, beach and lifestyle who tend to produce more internationally minded startups. Madrid’s startups have predominantly been Spain or Latin America-focused, but have become increasingly international in nature. Although not part of this survey, we expect Valencia to join next year, as city authorities have been going all-out to attract entrepreneurs and investors. The overall Spanish ecosystem is generally less mature than those in the U.K., France, Sweden and Germany, but it has been improving at a fast clip. More recently, entrepreneurs in Spain have moved away from emulating success in pursuit of innovative technologies. Following the financial crisis, the Spanish government supported the creation of startups with the launch of FOND-ICO GLOBAL, a €1.5 billi...

The Transformation Edge: Why AI is forcing the CFO and CHRO to rewrite the enterprise together

There was a time when the org chart told you how the company worked. Finance managed capital. Human resources managed people. Information technology managed systems. Operations managed workflows.  AI is blowing that model apart, creating a new discipline of “workforce economics,” where human talent, digital labor, technology investment and productivity increasingly have to be managed […] The post The Transformation Edge: Why AI is forcing the CFO and CHRO to rewrite the enterprise together appeared first on SiliconANGLE . from SiliconANGLE https://ift.tt/bDxK4Fn via IFTTT