Databricks makes bringing data into its ‘lakehouse’ easier

Databricks today announced that launch of its new Data Ingestion Network of partners and the launch of its Databricks Ingest service. The idea here is to make it easier for businesses to combine the best of data warehouses and data lakes into a single platform — a concept Databricks likes to call ‘lakehouse.’

At the core of the company’s lakehouse is Delta Lake, Databricks’ Linux Foundation-managed open-source project that brings a new storage layer to data lakes that helps users manage the lifecycle of their data and ensures data quality through schema enforcement, log records and more. Databricks users can now work with the first five partners in the Ingestion Network — Fivetran, Qlik, Infoworks, StreamSets, Syncsort — to automatically load their data into Delta Lake. To ingest data from these partners, Databricks customers don’t have to set up any triggers or schedules — instead, data automatically flows into Delta Lake.

“Until now, companies have been forced to split up their data into traditional structured data and big data, and use them separately for BI and ML use cases. This results in siloed data in data lakes and data warehouses, slow processing and partial results that are too delayed or too incomplete to be effectively utilized,” says Ali Ghodsi, co-founder and CEO of Databricks. “This is one of the many drivers behind the shift to a Lakehouse paradigm, which aspires to combine the reliability of data warehouses with the scale of data lakes to support every kind of use case. In order for this architecture to work well, it needs to be easy for every type of data to be pulled in. Databricks Ingest is an important step in making that possible.”

Databricks VP or Product Marketing Bharath Gowda also tells me that this will make it easier for businesses to perform analytics on their most recent data and hence be more responsive when new information comes in. He also noted that users will be able to better leverage their structured and unstructured data for building better machine learning models, as well as to perform more traditional analytics on all of their data instead of just a small slice that’s available in their data warehouse.

from TechCrunch https://ift.tt/38XI01I
via IFTTT

Comments

9 VCs in Madrid and Barcelona discuss the COVID-19 era and look to the future

Spain’s startup ecosystem has two main hubs: Madrid and Barcelona. Most observers place Barcelona first and Madrid second, but the gap appears to close every year. Barcelona has benefitted from attracting expats in search of sun, beach and lifestyle who tend to produce more internationally minded startups. Madrid’s startups have predominantly been Spain or Latin America-focused, but have become increasingly international in nature. Although not part of this survey, we expect Valencia to join next year, as city authorities have been going all-out to attract entrepreneurs and investors. The overall Spanish ecosystem is generally less mature than those in the U.K., France, Sweden and Germany, but it has been improving at a fast clip. More recently, entrepreneurs in Spain have moved away from emulating success in pursuit of innovative technologies. Following the financial crisis, the Spanish government supported the creation of startups with the launch of FOND-ICO GLOBAL, a €1.5 billi...

How to Stay Creative and Keep SEO in Mind

Information Technology Blog - - How to Stay Creative and Keep SEO in Mind - Information Technology Blog Search engine optimization (SEO) refers to customizing your website’s content to ensure that web browsers give your website a high SEO score. The sites with the highest SEO scores are featured on the search engine’s first page of search results for relevant searches. 71% of the click-throughs happen with articles listed on the first page of results on the search engine. This means that if your website’s article is the second (or third, or fourth page), it’s less likely the search user will even see your article. You want your article to be ranking as close to the top of the first page of results as possible. In order to have a good SEO score your site’s content needs to feature keywords and relevant phrases. It must be optimized for easy navigation between pages. It also needs to be referenced via external links that drive traffic to your site. Incorporating all of t...

Everything we know about HHS Protect, a secretive government project with Peter Thiel's Palantir that helps brief Trump's coronavirus task force

A secretive project at the US Department of Health and Human Services is working with technology companies to collect and analyze data related to the novel coronavirus . Dubbed "HHS Protect," the effort tracks information from around the country about coronavirus case numbers, hospital capacity, and even supply chain issues. HHS uses Palantir Technologies , a data firm cofounded by Peter Thiel, to distill that information for the White House coronavirus task force. Visit Business Insider's homepage for more stories . A secretive project at the US Department of Health and Human Services is working with technology companies to collect and analyze data related to the novel coronavirus. Dubbed "HHS Protect," the effort includes roughly 2.5 billion pieces of data from healthcare providers, government officials, and labs around the country about coronavirus case numbers, hospital capacity, and even supply chain issues. The goal is learn about the progress...

FREE INFORMATION TECHNOLOGY BLOG FOR PROFESSIONALS

Search This Blog