Platform 

• You can now work with non-notebook files in Databricks ( py, md,csv, txt, log files). For more information : https://docs.databricks.com/files/workspace.html
• Databricks runtime 12.2 standard and ML are GA
• The SQL admin console has been combined with the general Admin console to create a Unified Experience for admin users.
• Support for Jupyter notebooks is available in Repos. For more information: https://docs.databricks.com/repos/repos-setup.html#ipynb-repos
• Reload4J is replacing log4j in Databricks Runtime Versions 10.4 and earlier
• Terraform provider updated to version 1.12.0
• Databricks Runtime 13.0 and Databricks Runtime 13.0 ML are now available as BETA. For more information : https://docs.databricks.com/release-notes/runtime/13.0.html
• New metadata column fields denoting file block start and length
• New h3_CoverAsH3 and H3_CoverH3String are available for Geospatial processing when Photon is enabled ( DBR 13 required)
• All non-empty %sql results in notebooks are accessible using _sqldf.
• Databricks now supports selecting c7g instances during cluster creation on AWS. C7g instances do not support Photon runtimes.
• You can now run SQL cells in Databricks notebooks in parallel while attached to an interactive cluster. For more information : https://docs.databricks.com/notebooks/notebooks-code.html#sql-run-now
• The file editor has been updated to bring many of the notebook features to your file editing experience. Additionally, you now can directly execute Python, SQL, R, and Scala files from the file editor. For more information: https://docs.databricks.com/notebooks/notebook-editor.html
• Databricks account console now supports IP access lists to control access to the account console by IP address ranges. For more information : https://docs.databricks.com/security/network/ip-access-list-account.html
• Dolly is a cheap-to-build LLM that exhibits a surprising degree of the instruction following capabilities exhibited by ChatGPT. Databricks is open sourcing a simple Databricks notebook that you can use to build Dolly yourself on Databricks

Delta Lake

• Create table like feature for Delta tables ( DBR13 Required)

Governance

• Authenticate to Power BI and Tableau using Oauth.
• You can view the most frequent queries and users of any table registered in Unity Catalog. For more information : https://docs.databricks.com/data/table-insights.html

Workflows

• You can view lineage information for your jobs in the Databricks Jobs UI. For more information: https://docs.databricks.com/workflows/jobs/jobs.html#view-lineage

Databricks SQL

• Any rows are deleted when running data manipulation language (DML) operations such as DELETE, TRUNCATE, and replaceWhere with partitioned predicates.
• Databricks  SQL Serverless is GA in the supported region in AWS. For more information : https://docs.databricks.com/resources/supported-regions.html
• SQl Warehouse’s APi is getting a minor change or the field enable_serverless_compute, previously the default was true if serverless SQL endpoints were enabled for the workspace. The default is now false for most workspaces. For more information : https://docs.databricks.com/sql/api/sql-endpoints.html
• File, Query and feature store lineage are available
• You can format cell values in the new notebooks pivot table
• Non-UTF-8 characters are now supported in tables and visualizations

Machine learning

• Model serving previously called SRTI is now GA. It provides highly available and low latency services for deploying models . For more information: https://docs.databricks.com/machine-learning/model-serving/index.html
• In Unity Catalog enabled workspaces on a cluster running Databricks  13 ML or above you can publish both workspace and Unity Catalog tables to CosmosDB online Store.
• you can perform distributed training on PyTorch ML models using TorchDistributor (ML 13 required). For more information : https://docs.databricks.com/machine-learning/train-model/distributed-training/spark-pytorch-distributor.html