Note
This technote is a work-in-progress.
1 Abstract¶
Description of the L3 milestone for early DRP type processing.
2 Introduction¶
This document serves as a description of the milestone USDF ready for ComCam processing as well as a report on testing to achieve the milestone. The operations milestone ticket for this is PREOPS-1667. The description is in Section 3 while the test status is in Section 4.
3 Milestone Description¶
Annual Data Release Processing is planned to be a multi-site activity with the facilities in France, the UK and the US, wherein all data taken to date are reprocessed withe the latest algorithms and calibrations.
- The elements of a DRP are:
Distribution of raw input data to all sites
Distribution of processing graphs
Registration of output products (in the butler and Rucio)
Distribution of output products back to the USDF for the central archive and to the other Data Facilities as needed.
Demonstrate processing at a sufficient rate
It is anticipated that the Rucio and FTS tools will be used for data distribution and PanDA for distributed workflow. All three servers should be in production at the USDF.
4 Tests and Test Status¶
4.1 Distribution of raw input data to all sites¶
4.2 Distribution of processing graphs¶
4.3 Registration of output products¶
4.4 Distribution of output products¶
4.5 Demonstrate processing at a sufficient rate¶
See the reStructuredText Style Guide to learn how to create sections, links, images, tables, equations, and more.