Pipelet is a free framework allowing the creation, execution and browsing of scientific data processing pipelines.
- easy chaining of interdependent elementary tasks;
- web access to data products, to ease comparisons and traceability;
- a way to carry pipelines source code from development to production and to adapt to different hardware and software architectures.
Both engine and web interface are written in Python.
APCScheduler is a Python library which can sumit jobs to a Grid distributed system (gLite/WMS) from a User Interface (UI) which has access to a Storage Element (SE).
APCScheduler automatically creates a tar-ball of the binaries and handles data availability using a 'mailbox' repository on the Computing Element (CE). SE contents management, tar-ball creation and job submission can be done through some basic APCgrid unix commands.
APCScheduler also handles CC-IN2P3 and local job submission with the same syntax. Moreover, it handles job dependencies (submission of job group, job waiting, etc).
APCScheduler is easy to install as it consists of a single Python library plus some scripts.