Platform
- Databricks Runtime 11.3 and 11.3 ML are in GA
- User settings, Admin Console, Manage Account, Help, Workspace Switcher and search have been moved from the sidebar to the top bar in the UI
- Improved inference of Data columns in CSV Files
- Enable admin protection for no isolation Shared clusters. It helps protect admin accounts from sharing internal credentials in any environment
- Personal compute policy allows users to easily create a single machine compute resources for their individual so they can start running workloads immediately. See more information : Link
- Add Data portal provides access to common data sources configurations and file upload UIs .
- Format Python Cells in Notebooks
- Admins can enable Enforce User Isolation to prevent creating or starting a cluster with “ No Isolation Shared”
- Databricks kinesis connector now supports reading from Kinesis Data Streams in EFO mode
- Connect to Fivetran from the Add data Portal
- Partner connect supports connecting to erwin Data modeler
Governance
- Unity Catalog now supports privilege inheritance. It means that granting a privilege on a catalog or schema automatically grants all the privileges to all current and future objects.
Delta Lake
- Clone support for Parquet and Iceberg ( Shallow and Deep Clone)
- Delta lake stats collection schema now matches column order in table schema definition
Databricks SQL
- Serverless SQL Warehouses are now available in regions eu-central-1 and us-east-2
- Global search is now available in the SQL persona. You can now search for queries, dashboards and alerts with global search in any persona by name.
- You can now find the query progress bar in the footer and the editing visualization action in the kebab menu
- Autocomplete now supports Delta time Travel and provides column autocomplete when defining a foreign key
- The row limit downloading query results to Excel has been increased from 64k to 100k rows.
- A new Warehouse type, Databricks SQL Pro is introduced to Databricks SQL. It enables the use of Workflows integration, query federation, geospatial features and predictive IO.
- A New Serverless region is available eu-west-1
Security
- Databricks is a FedRamp authorized cloud service offering at the moderate impact level.
- You have the ability to make some AWS privateLink workspace updates by using the admin console
- AWS PrivateLink support is now generally available.
- IP Access Lists features apply only to requests over the public addresses.
Machine Learning
- AutoML supports the use of existing Feature Store feature tables in AutoML experiments.
- AutoML supports DecimalType