Skip to content

Stop of job leads to SchedulerException

On stop Loadable kernel modules sample job I faced with SchedulerException:

Future task '9': join method returned 'None'.
Future task '9': process 1558588 joined, going to check its exit code
Future task '9': exit code of the process 1558588 is '0'
Future task '88ea2535-8040-4536-88ca-7021c1b57830': join method returned 'None'.
Future task '88ea2535-8040-4536-88ca-7021c1b57830': process 1543332 joined, going to check its exit code
Future task '88ea2535-8040-4536-88ca-7021c1b57830': exit code of the process 1543332 is '255'
2024-04-11 16:38:15,142 (native.py:453) root  INFO> Future processor of job 88ea2535-8040-4536-88ca-7021c1b57830 returned 255
2024-04-11 16:38:15,142 (runners.py:329) root ERROR> Job 88ea2535-8040-4536-88ca-7021c1b57830 has failed: SchedulerException('Exited with exit code: 255 (please, inspect unknown reports and logs)')
2024-04-11 16:38:15,142 (native.py:402) root  INFO> Going to cancel execution of the task 9
2024-04-11 16:38:15,143 (native.py:414) root WARNING> Cannot terminate process 1558588: [Errno 3] No such process
2024-04-11 16:38:15,147 (native.py:453) root  INFO> Future processor of task 9 returned 0
2024-04-11 16:38:15,148 (native.py:402) root  INFO> Going to cancel execution of the task 10
2024-04-11 16:38:15,148 (native.py:414) root WARNING> Cannot terminate process 1558579: [Errno 3] No such process
2024-04-11 16:38:15,152 (native.py:453) root  INFO> Future processor of task 10 returned 0
2024-04-11 16:38:15,220 (resource_scheduler.py:204) root  INFO> Submit information about the workload to Bridge

On stop validation Bugs found by Klever job I faced with uncaught exception:

2024-04-10 18:16:20,054 (__init__.py:103) root ERROR> Uncaught exception
Traceback (most recent call last):
  File "/home/druidos/development/clean/klever/venv/bin/klever-native-scheduler", line 33, in <module>
    sys.exit(load_entry_point('klever', 'console_scripts', 'klever-native-scheduler')())
  File "/home/druidos/development/clean/klever/klever/scheduler/main.py", line 57, in native_scheduler
    scheduler_impl.launch()
  File "/home/druidos/development/clean/klever/klever/scheduler/schedulers/__init__.py", line 209, in launch
    self.runner.cancel_job(identifier, self._jobs[identifier],
  File "/home/druidos/development/clean/klever/klever/scheduler/schedulers/runners.py", line 335, in cancel_job
    self.cancel_task(task["id"], task)
  File "/home/druidos/development/clean/klever/klever/scheduler/schedulers/runners.py", line 360, in cancel_task
    item["status"], _ = self._cancel_task(identifier, item["future"])
  File "/home/druidos/development/clean/klever/klever/scheduler/schedulers/native.py", line 286, in _cancel_task
    return self._cancel_solution(identifier, future, mode='task')
  File "/home/druidos/development/clean/klever/klever/scheduler/schedulers/native.py", line 415, in _cancel_solution
    return self._postprocess_solution(identifier, future, mode)
  File "/home/druidos/development/clean/klever/klever/scheduler/schedulers/native.py", line 464, in _postprocess_solution
    with open(results_file, encoding="utf-8") as fp:
FileNotFoundError: [Errno 2] No such file or directory: 'scheduler/tasks/17/decision results.json'