At its annual event in Las Vegas, Snowflake announced the availability to all public preview customers of Snowpark for Python, announced in November. With this framework, the publisher ultimately wants to work with streaming data. Still with the goal of improving data access, the publisher also introduced Iceberg Tables.
Snowflake opened its annual conference on June 14 in Las Vegas with a variety of announcements to complement its programmable and scalable data store platform suitable for a wide variety of workloads. Through its services, the supplier seeks to be close to data scientists, data engineers, and application developers and best meet their needs. In this sense, Snowflake announced to extend native Python support and launched Snowpark for Python, which is now available for all customers in public preview. This addition improves accessibility and programmability for the professions mentioned above. Actually, Snowpark for Python is natively integrated to help companies build data -driven applications. This merger became possible following the acquisition of start-up Streamlit (in March 2022).
On the second day of the event, Torsten Grabs, Director of Product Management discussed having a public preview of Snowpark for Python. (Credit: CS)
The Snowpark development framework, introduced in June 2021 and available to everyone since the end of January, helps developers build pipelines and scalable applications as well as machine learning flows directly to the platform using their favorite languages and libraries. . The editor defined extending access to data to be able, in the long run, to work with data seamlessly. Currently in private preview, this tool dedicated to streaming data support called Snowpipe Streaming can significantly improve latency. Christian Kleinerman, Senior Vice President, Product, Snowflake, commented on this point: “Snowpipe Streaming will deliver 10 orders of magnitude improvement in latency over what exists today, from minutes to seconds.” Christian Kleinerman also announced a massive investment in this area, considering the “ease of use of continuous processing [comme étant] a major issue.
Materialized Tables and Iceberg Tables released
A feature was also presented in a keynote by Christian Kleinerman, called Materialized Tables, intended to support all Snowflake queries. “Typically, it will take a lot of effort to set up separate streaming systems that connect to Snowflake’s reference data using complex imperative programming. But Materialized Tables, which are currently being developed, eliminates complexity and simplifies the integration and modification of streaming data ”explains Christian Kleinerman. He defined this as “the materialized table is a declarative data pipeline that works on only a few lines of SQL”.
In the second step, the publisher announced that it plans to extend its external tables to support Apache Iceberg so that users can access data from on-premises storage systems directly from the cloud and disclose Iceberg Tables.
Christian Kleinerman, senior vice president of product for Snowflake, announced Iceberg Tables. (Credit: Snowflake)
Presented as “first class” tables, the latter “offers the same management, DML and CRUD functions as internal tables, with similar performance.” Emphasizing interoperability, he explained that Iceberg tables use open metadata and the Parquet open file format within customer storage buckets. “I think PPE is we need to expand the Data Cloud to different systems. We’re starting with two of our big partners, Dell Technologies and Pure Storage, which are the biggest integrations and will be launched in the coming weeks,” he said. end by Christian Kleinerman. Note that partnerships formed with these two storage providers make it possible to provide local analytical or cloud processing using Snowflake data store tools.