A small subset of the data warehouse extraction work involves running about 20 programs. Currently, they are being run serially from a .bat script. Overall, there are probably 150+ things that need to be run by adTempus.
SET PROG_LIST=^
prog_00.exe ^
prog_01.exe ^
prog_02.exe ^
...
prog_18.exe ^
prog_19.exe ^
prog_20.exe
FOR %%p IN (%PROG_LIST%) DO (
%%~p 2>\logdir\%%~p.log
)
I would like to create separate adTempus jobs for each of these. The goal is to be able to run two or more concurrently. How "concurrent" would depend on the resources available on the system. There might be a small, 4 CPU, system for development or test, but a 12 CPU system for production.
I realize that load-based scheduling is not yet available in adTempus. I would like to know if there is some way I could decide on hardcoded concurrency now that could easily be managed as system resources scale?
One way to do this might be to have a job that starts programs 0, 5, 10, and 15. These, in turn, would be grouped into (0,1,2,3,4), (5,6,7,8,9)... with these running serially from responses. As the system scaled up, perhaps groups of three (3) would be possible to get faster completion.
For what I have been doing, this seems like an extremely difficult, time intensive way to manage concurrency. Is there a better way in adTempus 4.2 to do this?
Staff
