Last year I started using Azure WebJobs to run some background processing for a website. At the time, I was not using the WebJob SDK at all, I was just writing console apps that blocked and stayed in a loop … and most of the time everything worked great.
Every now and then, I would notice the webjob restarting or being in some status other than “running” or “stopped”. Most of the time the problem ended up being my code - things like data scenarios that weren’t accounted for in the code or maybe config settings that I didn’t update before deploying … you know dumb stuff that you catch pretty quick. Solving the problems that occurred right after I deployed new jobs was pretty easy. However, it wasn’t as easy to know if a problem occurred at some other point in time so I could address the problem proactively.
If a WebJob goes down and no one is looking at the portal to notice … did it really go down?
The answer of course is: yes! But if the webjob is down, how could you have it tell you it is down? Seems like its sort of a chicken and egg problem. The first solution that came to mind was a dead man switch that pinged a heartbeat every so many seconds or minutes to let me know if it is a live. Then of course I would need to write the monitoring code to check for the updates and notify me when they are outside of the expected time limits. All doable … though I didn’t really want to write a monitoring system.
Turns out, Microsoft has already solved this for you with web tests.
My initial exposure to web tests were the ping tests that you can setup to call your site from different data centers in order to test for a specific status code and/or text in the result. Very useful stuff for monitoring websites. A good overview of web tests can be found here: Monitor availability and responsiveness of any web site.
Problem: The webjob doesn’t expose a URL so how can I setup a web test?
Solution: the WebJobs API and the multi-step web test.
On the WebJobs API github page, towards the bottom - you will find the apis for continuous jobs. The api that I needed was just a simple GET call.
As long as I am logged into the azure portal, I can access the WebJobs api off of the Kudu url: http://<yourwebsite>.scm.azurewebsites.net/api/continuousjobs/<jobname> to get the webjob’s current information -> included the status.
Next problem: How do you get the web test to be logged into the azure portal in order to call the WebJobs api?
This is where the multi-step web test comes in. One of the cool things about the Kudu api is that is also supports basic authentication – which means the call to the api and authentication can all be done in a single http call. In order to get the Authorization header set, you need to create a Web Performance Test.
Here are the steps to create a Web Performance Test using Visual Studio
1. Open Visual Studio, File -> New Project
2. In the search box, type performance
3. Select the Web Performance and Load Test project template
4. Give the project whatever name you want (you don’t really need to project – just the .webtest file)
5. Click OK
Once the project loads, you should have the WebTest1.webtest active in the editor
Create the request
6. In the editor, right click on the WebTest1 node and select Add Request
7. In the properties window, set the following properties (make sure you replace the values in the url with your values):
Expected HTTP Status Code = 200
Url = https://<yourwebsite>.scm.azurewebsites.net/api/continuousjobs/<jobname>
Add the basic authentication header
8. In the editor, right click on the request node and select Add Header
9. In the properties window, change the Name to Authorization
The username and password you use for basic authentication with Kudu is the same username/password you use for FTP or deployment. This information can be found and reset in the Web App –> Settings -> Deployment credentials blade in the portal.
The value of the Authorization header is “Basic <base64encodedString>”. The things to note here are: the prefix “Basic” with a space after it and the base64 encoded string of <username>:<password>.
Header value Example:
For the username: simplewebjob1 and a password of “password1234$”. Base64 encode the string “simplewebjob1:password1234$” to get c2ltcGxld2Viam9iMTpwYXNzd29yZDEyMzQk
Now add the prefix to get the final header value of “Basic c2ltcGxld2Viam9iMTpwYXNzd29yZDEyMzQk”
10. Set the Value to your basic authentication header value (you can encode your username password here on my utilities page).
Add the validation to look for “status”:”Running”
11. In the editor, right click on the request node and select Add Validation Rule…
This will open the Add Validation Rule dialog
12. Select the Find Text rule
13. Set the Find Text property to the following:
14. Click OK.
The editor should look something like the following when you are done:
Now save and run the test in Visual Studio to verify it works.
This WebTest1.webtest file can be used to create a multi-step web test in the Azure portal. If you need help on doing that, a great walkthrough how to setup the web test in the portal is about half way down this article: Monitor availability and responsiveness of any web site.