Jacob Quinn

Data scientists need to work with various data sources and sinks in their projects. During the workshop I you will learn how you can work with standard data formats using DataFrames.jl. A special focus will be put on working with data that is larger than available RAM.

Data science pipelines created in Julia typically need to be integrated into larger workflows involving various tools and technologies. Therefore an important aspect is ensuring interoperability, especially for the case of large data that does not fit in RAM of a single machine.

During the workshop we discuss working with the following data formats:

Section 1: https://github.com/bkamins/JuliaCon2023-Tutorial

* statistical packages (Stata/SAS/SPSS and RData);
* databases (SQLite and DuckDB);
* Apache Parquet.

Section 2: https://github.com/quinnj/JuliaCon2023-Tutorial

* CSV;
* Apache Arrow;
* JSON.

The examples will DataFrames.jl that provides a representative implementation of Tables.jl table.

32-144

Bogumił Kamiński

Working with DataFrames.jl beyond CSV files

The objective of this minisymposium is to gather people interested in working with tabular data to allow them to discuss various aspects of the Julia ecosystem in this area.

The list of talks is scheduled as follows:

Session 1 (10:30-11:30):

* What is new in DataFrames.jl - highlights of 1.4, 1.5, and 1.6 releases. (Bogumił Kamiński)
* Parallelizing your data with Dagger and DTables (Julian Samaroo)
* Working with geographical data using DataFrames.jl (Przemysław Szufel)

Session 2 (11:30-12:30):

* Ingesting larger data in Julia: evolution of JSON (Jacob Quinn)
* Tips & tricks for printing tables using PrettyTables.jl. (Ronan Arraes J. Chagas)

Session 3 (14:00-15:00):

* Tidier.jl: Bringing the Tidyverse to Julia (Karandeep Singh)
* Timeseries data manipulation using TSFrames.jl. (Chirag Anand)

Geometric Algebra at compile-time with SymbolicGA.jl

Tools and techniques of working with tabular data

During the BoF we want to discuss future directions of development of JuliaData ecosystem. The objective is to identify priorities of development of current packages in the ecosystem and discuss what functionalities are missing and should be added in new packages.

Selected planned topics for discussion are:

* The state and future of integration databases (both SQL and NoSQL);
* Creation, execution, and monitoring of data processing pipelines; especially considering containerization and deployment in the cloud (including discussion of integration with standard tools available in this domain);
* Do we need/can benefit from GPU acceleration for processing tables?
* Scaling computations (out-of-core processing, supercomputer processing): what is needed, what is the current status? What should be the roadmap for Dagger.jl and DTables.jl?
* How can we make it easier for users to move from R/Python to Julia?

Jacob Quinn

Talks:

Working with DataFrames.jl beyond CSV files

Tools and techniques of working with tabular data

Future of JuliaData ecosystem

Platinum sponsors

Gold sponsors

Silver sponsors

Bronze sponsors

Academic partners

Local partners

Fiscal Sponsor