Skip to main content

Your first Workflow Run

We are finally there, we can run our first workflow. In this section we will run our first workflow, that we designed in the previous section. Remember it will take an image , max project it, threshold it and then measure the fraction of the image that is below the threshold, to estimate the fraction of the image that is not covered by cells. In this section you will learn:

  • How to reserve a workflow
  • Where to look when your workflow is not available
  • How to run a workflow
  • How to monitor the progress of a workflow
  • How to inspect the result of a workflow

How to reserve a workflow​

Lets start by reserving our workflow. For this we will navigate to the Dashboard tab. Search our Workflow title (which is now a Node) and right-click on the "Reserve" button. This will reserve our workflow for us.

Pretty straight forward right? Its the same as reserving a normal node, but now we are reserving a workflow. You will notice however that at the end of the reservation we illustrate another feature of the Platform. The user clicked on the reservation and then after clicking on the first provision we were transported to the "Monitor Page":

A workflow includes nodes, and these nodes are the "dependencies" of the workflow. In order to run a workflow we need to make sure that all the nodes are available and ready to be used. When we reserve a workflow, the Scheduler will in turn reserve all the nodes that are part of the workflow. And then it will wait until they are available. This is what we see in the "Monitor Page". Here we can see the status of all the nodes that are part of the workflow. And we can see if they are ready to be used (green) or not (red). If they are not ready to be used we can click on the node and see why they are not ready to be used. Most of the time this means you have to start or restart the linked application.

The Monitor Page

The monitor page illustrates an important concept of the platform. As Arkitekt with a distributed system, we need to keep track of what can run when. Workflows are a perfect example of this. In order to run a workflow we need to make sure that all the nodes are available. This Dependency Graph Management is a core feature of the platform, and will find its explaination in the Provision and Reservation section. Btw, in the next generation of rekuest, we will make this a lot more intuitive. That's the hope at least.

Run our analysis​

And now we can finally run our analysis. Lets start by navigating to the "Data" tab. Lets use the same image as we used in the previous section.

  1. Navigate to our data in the Data tab

Lets inspect our data in the "Data" tab. We should see our data in the "Data" tab. (If you are doing this part of the tutorial, on another day you might need to open the "Home", sidebar and step back one day to see your data.)

  1. Drag and Drop or right click the image the @mikro/representation we want to convert

This time in our Action menu we will see a new option. "Analysis Run", this is our first Workflow Action. Lets drop on it (or click it).

  1. You are prompted with a dialog.

This dialog is our first encounter with an Arkitekt Assign dialog. Here Arkitekt will ask us to confirm the assignment.

  1. This is new!

You might notice that you are immediately transported to the Run page, where you can inspect the progress and result. Its a bit fast. But you can always go back in time. Just click on the live button in the bottom left, corner. And you will be transported back in time and you can scan through the history of your run (current bug, you might need to refresh the page (right clikc -> refresh) to scan)

The Run Page

Everytime you run a workflow, Arkitekt will create a Run for you. This run will keep track of the progress of your workflow, and will also keep track of the result of your workflow. A run is not only an indicator of the live progress of your workflow, but will also be your go to point to inspect the processing pipeline and potential debug entrypoint. All data is linked in your run and can be exported (e.g. in your methods section just like here). You can find your latest runs in the Workflow tab.

When double clicking on the node in replay or live mode in a run, you can inspect the data flowing through the graph. Not to helpful here, but very much so in the advanced workflows.

Lets inspect our image​

But lets not only look at our workflow. Just like we did in the previous section, we can also inspect our image. Lets start by clicking on the Data tab.

While you click on your image, you will now see that there is derived images that are linked to your image. These are the images that are created throughout the workflow. Just explore this datagraph. You can click on the images and inspect them.

The Data Graph vs the Provenance Graph

Interestingly there is two types of graphs that are linked to your data. The Data Graph and the Provenance Graph. The Data Graph is the graph that is created through interlinking of data by the application itself (e.g. a thresholding algorithm can decide to declare that the thresholded image is a derivative of the original image). The Provenance Graph is however a graph that is created through keeping track which Node task created an image. If you click on the Provenance icon you will see a reference to the Assignation (i.e. the Task) that created that image. And that task will in turn have a reference to the Workflow Assignation that created it.