Pipelet is a free framework allowing the creation, execution and browsing of scientific data processing pipelines.

It provides:

  • easy chaining of interdependent elementary tasks;
  • web access to data products, to ease comparisons and traceability;
  • a way to carry pipelines source code from development to production and to adapt to different hardware and software architectures.

Both engine and web interface are written in Python.

Getting Pipelet

Pipelet documentation



APCScheduler is a Python library which can sumit jobs to a Grid distributed system (gLite/WMS) from a User Interface (UI) which has access to a Storage Element (SE).

APCScheduler automatically creates a tar-ball of the binaries and handles data availability using a 'mailbox' repository on the Computing Element (CE). SE contents management, tar-ball creation and job submission can be done through some basic APCgrid unix commands. 

APCScheduler also handles CC-IN2P3 and local job submission with the same syntax. Moreover, it handles job dependencies (submission of job group, job waiting, etc). 

APCScheduler is easy to install as it consists of a single Python library plus some scripts.   

Getting APCScheduler

APCScheduler documentation