Skip to main content

Job Time Limits

fsched allows users to specify the maximum runtime when submitting jobs. After the time limit is exceeded, the scheduler will forcibly terminate the job.

Partition Default Time Limit​

Administrators can set a default job time limit on a partition. For example, you can set the default time limit for all jobs in a partition to 1 hour. Then all jobs submitted to that partition are subject to this limit. The partition default time limit uses the partition configuration parameter DefaultTime when the user does not specify a time limit. Supported formats:

  • Minutes
  • Minutes:Seconds
  • Hours:Minutes:Seconds
  • Days-Hours
  • Days-Hours:Minutes
  • Days-Hours:Minutes:Seconds
  • UNLIMITED

Note: When specifying seconds, the actual limit may exceed the value, depending on controller load, possibly by several minutes.

Partition Maximum Time Limit​

A partition can specify a maximum time limit. Users cannot exceed this limit when submitting or modifying job time limits. This parameter is MaxTime.

Supported formats:

  • Minutes
  • Minutes:Seconds
  • Hours:Minutes:Seconds
  • Days-Hours
  • Days-Hours:Minutes
  • Days-Hours:Minutes:Seconds
  • UNLIMITED (default)

User-Defined Time Limit​

If users want to adjust a job's time limit, they can use scontrol update job to modify the job's TimeLimit field, with a minimum unit of minutes. For example, to set the time limit to 2 hours:

scontrol update job <jobid> timelimit=02:00:00

By default, users can only shorten the time limit and cannot extend it. If users want to extend the time limit, set SlurmctldParameters to allow_user_incr_time in the global configuration. Then users can extend the TimeLimit field via scontrol update job, but still cannot exceed the partition maximum time limit.

Note: If fsched is managed by FCP / FCCE, set: SlurmctldParameters=cloud_dns,nohold_on_prolog_fail,allow_user_incr_time

timelimit supports the following formats:

  • Minutes
  • Minutes:Seconds
  • Hours:Minutes:Seconds
  • Days-Hours
  • Days-Hours:Minutes
  • Days-Hours:Minutes:Seconds