Job Queue Management

Holistics Job Queue Management#

This section talks about Holistics Job Queue system, or how jobs are being distributed and managed from within Holistics.

A job is an operation that usually takes a long time, and perform some business logic. Examples:

  • When a user view a report, a report job is sent to backend for execution
  • When an email schedule invokes, an email job is created
  • When a data import runs, a data import job is created

Job Queues#

Your whole organization has a limit on number of jobs running at the same time. This is to protect both your databases and stability of Holistics system.

Furthermore, depending on the nature of the job, it'll be classified into different queues (or pools). For example a report job runs on a different queue than a data transform job.

Please contact us if you want to adjust the number of queue sizes of each queue.

Limit Number of Report Jobs Run Per User#

Since the max concurrent jobs are being capped per queue, when a user A purposely or accidentally farms multiple jobs, these jobs quickly take up all the available slots in the queue, causing other users unable to use the system.

To prevent this from happening, admins can set a limit on how many report jobs each user can run concurrently. This is done in the Admin, Settings page.

Cancel Running Jobs#

Some type of jobs are able to be cancelled while running. This is incredibly helpful when a user accidentally farms a long-running job as an available slot in the queue will be wasted.

A running job's ability to be cancelled is determined based on job's Source type and Action. Currently, Holistics supports cancelling these type of running job:

  • DataTransform: execute
  • DataImport: execute
  • EmailSchedule: execute
  • QueryReport: execute, preview
  • DashboardWidget: execute

For QueryReport jobs (execute, preview), there are also Cancel buttons in Report view and Report editor, which share the same effect of cancelling the job from Jobs monitoring screen.

Avoid Running Duplicate Report Query Jobs#

If 2 users (A and B) open the same report within short amount of time, there's a high chance a duplicate query will be sent to the system while the first query is still running. This unnecessarily overloads the system, and increases the waiting time of both users.

To avoid this happens, Holistics have a built-in de-duplication mechanism, works as follow:

  • Every time a query job is submitted, the query hash is used to look up concurrent running jobs within the last 10 minutes to find the same query currently being executed at customer database
  • If found, the job status is set to "already existed" and the job result is routed to the previous running job with the same query

If the first job already exists and the result is stored in cache, the caching mechanism will kick in. The second query will not be sent at all, and we will use the cached result to serve to user B.

Cancel Running Queries#

While cancelling a running job, Holistics will also try to cancel all job's running queries in order to save your database server's resources.

Holistics uses a simple, yet effective mechanism to cancel job's running queries. We includes the job's id as a comment in every query sent to your database server, and uses this information to identify the specific process running the query. Afterwards, Holistics will send a specific query depending on DataSource type (e.g, pg_cancel_backend for PostgreSQL) to kill the identified process.