Galaxy cloud

Galaxy is actively used in cloud computing. This page contains links and brief description of main Galaxy projects related with clouds.

CloudMan

CloudMan is a cloud manager that orchestrates all of the steps required to provision a complete compute cluster environment on a cloud infrastructure.

It is primarily used in the context of Galaxy cloud and CloudBioLinux.

It allows one to manage the cluster, all through a web browser.

It makes possible to customize each instance of CloudMan and, if desired, preserve those customizations [1].

It provides support for instance sharing. Each CloudMan instance can be shared as a point in time configuration (in terms of tools, data, and configurations) with individuals or made public.

Implementation

Python standalone web application, MIT license.

CloudMan supports:

• Amazon Web Services - EC2, EBS
• OpenStack
• OpenNebula

Instance customization

At the infrastructure level, each instance is composed of [2]:

• machine image - common denominator across all instances and contains all of the core software and libraries;
• configuration repository - each CloudMan instance has its own configuration repository (for AWS - Simple Storage Service (S3)). It includes:
  • source code for CloudMan itself
  • boot-time contextualization script
  • references to the persistent storage resources
  • any application-specific configuration files.
• one or more persistent storage units - store both data and applications available to the instance. They are realized as a combination of data volumes and snapshots.
  • for AWS - Elastic Block Store (EBS) volumes
  • for OpenStack - Nova volumes.

Tools that are used but not modified at runtime are stored on the snapshots. At instance runtime, those snapshots are used to create temporary volumes, which are attached to instances and used in read-only mode. These snapshots can be modified to include any desired tool. Such modifications are performed at the file system level and the process of adding a tool is the same as installing a tool on any other comparable system.

Once modified, a new EBS volume is created and the instance configuration in the persistent configuration repository simply needs to point to the new snapshot and the instance will use it as part of its deployment.

The process of persisting the custom cluster configuration has been integrated into CloudMan’s web interface, thus simplifying and automating this process.

Publications

Error fetching PMID 23181507:
Error fetching PMID 22700313:
Error fetching PMID 22068528:
Error fetching PMID 21210983:

Source code

Tool	URL	Comment
CloudMan	https://bitbucket.org/galaxy/cloudman/	Python, (82 files, ~800 kb)