AWS
Announcing PartiQL: One query language for all your data
AWS has open-sourced a new query language, based on SQL (“SQL-compatible”, as the blog post puts it). Essentially, it is a super-set of SQL that requires the JRE. This project bears watching, but will only be successful outside AWS if data store query engines add support for PartiQL. To further that aim, AWS is releasing the specification and reference implementation(written in Kotlin). The AWS services that support PartiQL thus far are:
- Amazon S3 Select
- Amazon Glacier Select
- Amazon Redshift Spectrum
- Amazon Quantum Ledger Database
Build highly available MySQL applications using Amazon Aurora Multi-Master
Not yet available for the PostgreSQL-compatible version of Amazon Aurora. Let’s hope it will be.
EBS default volume type updated to GP2
Per the announcement: “GP2 volumes offer lower latency and higher throughput than Standard volumes.”
Techtalk: Best Practices for Running Spark Applications Using Spot Instances on EMR
August 28. This is a 300-level session.
Techtalk: Best Practices for Transforming and Analyzing Data in Your Data Lake
August 21. A 200-level session.
PostgreSQL
PostgreSQL: Regular expressions and pattern matching
The author intends this to be the first in a series of posts on using regular expressions (RE) with PostgreSQL. This post overviews four RE operators.
Python
Python in Visual Studio Code – August 2019 Release
Visual Studio Code has been coming on strong as a popular cross-platform and cross-language IDE. This release furthers the support for Python and Jupyter notebooks.
Software Updates
DBeaver 6.1.4 (2019-08-04)
Among the fixes are these:
- PostgreSQL:
- Array data type handler was fixed
- Indexes metadata reading was fixed
- Execution plan visualization was fixed for CTE nodes
- SQL Server:
- Support of identity columns creation was added
- Session manager was fixed
- Oracle:
- Scheduled jobs metadata reading was fixed
- Session management was fixed for RAC mode
General Data-Related
DFLib – a lightweight, pure Java implementation of DataFrame
Dataframes (think database tables, spreadsheets) are foundational data structures in Python, R, and Spark. This library provides similar functionality when writing Java.
Liquibase Improving Community Support
In a sign of the continued strength of Liquibase as a community project, this post announces the hiring of an open source community manager.
This Week in Neo4j – Search Phrases in Bloom, SSIS Data Flow, Project and Libraries Dependency Graph
Traversing the Land of Graph Computing and Databases
Based on the author’s talk at Pycon X, this is a high-level overview of the resurgence of interest in graph-based technology and graph databases.
Upcoming Conferences of Interest
Strata Data Conference – New York, September 23-26
This is one of a series of O’Reilly-sponsored conferences on big data and data science. There are two coming up next year: Strata Data Conference – San Jose (March 15-18, 2020) and Strata Data Conference – London (April 20-23, 2020).
Classic Paper or Reference of the Week
Since I linked to a couple of blog posts above on the R “tidyverse”, I thought some folks might be interested in the paper by Hadley Wickham that started it all: Tidy Data, as published in the Journal of Statistical Software.
Data Technology of the Week
AgensGraph
An Apache 2.0 licensed-project that supports a property graph data model on top of PostgreSQL (version 10.3). AgensGraph supports both ANSI-SQL and openCypher for querying. There’s an enterprise version from Bitnine.
Metadata Standard of the Week
Friend of a Friend or “FOAF” is an ontology, using RDF and OWL, to describe persons, activities, and relations to other people and objects. An example use of FOAF is to describe a social network.