shepherd should enforce explicit time limits on how long a component is allowed to run for

[lincoln]

shepherd (engine/dog) should set explicit limits on how long a component is allowed to run for (wall-clock time). e.g. if a component it taking (say) >6 hours, stop it.

ideally this should be a .conf file per component - e.g. oztivo shouldn't take more than 5 minutes, sbsnews_grabber shouldn't take more than 1 minute, etc.

[max]

Category 2 grabbers should all complete within roughly the same time period, i.e. quite quickly. For Category 1 grabbers, you're basically talking about a "speed" rating. This could be used in grabber selection, too, to differentiate between a relatively fast C1 grabber and a relatively slow one. Although I'm not sure how you'd handle grabbers that are fast sometimes and slow other times (eg with/without Tor).

[anonymous]

One way to handle this would be to define a function in each grabber's config file that takes a grabber command line as argument and returns an upper bound on how long the grab could reasonably be expected to run. In most cases, the function would be trivial, ignoring the argument and return a constant. But it could look at the number of channels and days to be grabbed, whether or not tor is installed, and even the state of the cache if it wanted to. The advantage of using a function like this is that you could switch from a trivial constant definition to a more elaborate one without changing any other code.

[lincoln]

limits set via [271], enforced via [274]