At the University of Chicago, my work is focussed on accelarating science through simplifying and streamlining access to computational infrastructure. Since the plateauing of single core performance, exploiting parallelism is crucial to keep up with computing demands. To this end, I've explored parallel programming models, high-performance distributed systems and marrying the two through Python based libraries. A curated set of projects that fit this theme follow.
Parallel Scripting Library for Python.
Parsl is a Python library for programming and executing data-oriented workflows (dataflows) in parallel. Parsl scripts allow selected Python functions and external applications (called apps) to be connected by shared input/output data objects into flexible parallel workflows. Rather than explicitly defining a dependency graph and/or modifying data structures, instead developers simply annotate Python functions and Parsl constructs a dynamic, parallel execution graph derived from the implicit linkage between apps based on shared input/output data objects. Parsl then executes apps when dependencies are met. Parsl is resource-independent, that is, the same Parsl script can be executed on a laptop, cluster, cloud, or supercomputer.
Serverless Supercomputing.
funcX is a High performance function serving system designed to orchestrate scientific workloads on campus clusters, clouds and supercomputers.
funcX is currently in Alpha testing.
Enabling Secure and Scalable Data Analytics in the Cloud.
Cloud Kotta is a integrated solution for delivering web scale compute on data is a secure fashion. The infrastructure components that provide compute-nodes, storage-disks, databases etc are provided by Amazon Web Services, whereas a software and security layer built over these services create a simple, scalable computing solution. Cloud Kotta is this meta description of infrastructure that can be used to create an on-demand virtual datacenter on AWS.
An instance of Cloud Kotta is hosted by the KnowledgeLab at the Computation Institute. This instance is currently available here.
