MacAdmin

Moving Beyond Scheduled Jobs to Event-Driven Workflows

I have written before about some of the internal tools we are writing to try and automate certain tasks as well as serve as a bit of glue code between our internal systems. We tend to schedule these scripts to run either using Cron, or a personal favorite, jobber. Overall, this has been a tried and true approach, and we are in fact making great headway in automating several tasks, some smaller, some larger. However, these kinds of automated, scheduled jobs often come with unnecessary overhead. You can build in checks to not repeat operations if they have already occurred, but you still have to write and run the check itself, and when that means possibly iterating over thousands or tens of thousands of machines/records/files, etc., it starts to add up. In other circumstances you may not even write the check and perhaps just make the change over and over again, whether it was needed or not.

Event-driven Workflows

I had seen “Event-driven workflow” in several marketing campaigns for products up until this point, but never really understood what that meant. Turns out, it’s not too complex.  Essentially, “event-driven” or “events” really just mean HTTP callbacks, which are typically HTTP POST requests that the application makes upon an event happening. It means that instead of you having to ask the application a question, it can instead just provide an answer allowing you to instantly take action. Since we are working with AirWatch internally, we’ll be using that in our examples.

AirWatch has what they call “Event Notifications” (more on this later) which can “react” to events happening in your AirWatch environment, such as a device enrolling, or being deleted, or even just a device asset number change. Upon these tasks happening, AirWatch can trigger an Event Notification (an HTTP callback) and make an HTTP POST request to any defined URL. With Event Notifications, we no longer have to do the hard work of pulling all records or devices, and iterating over them looking for status changes or new devices, we can simply let the application tell us that something happened, and take the appropriate actions!

This is not to say scheduled jobs don’t still have their place in automation workflows, they most certainly do, but utilizing event-driven workflows can allow for far quicker reaction or responses to events, and save us some CPU time by only taking the actions needed at that time, for that event.

AirWatch’s Event Notifications

As mentioned above, AirWatch is what we now use internally, and lucky for us, it has Event Notifications built into the product. These can be found and enabled under:
Groups and Settings –> All Settings –> System –> Advanced –> API –> Event Notifications

Screen Shot 2018-05-10 at 11.45.54 AM.png

Once at that screen, click the Add Rule button and enter the URL that you want the notifications to be sent to (for testing, I used this handy little website http://webhook.site). You can also include authentication if the URL destination requires it. Then you choose the format you want the notification sent in, which I set to JSON, as I think it’s easier to parse and read.

Screen Shot 2018-05-10 at 4.04.15 PM.png

Once you have those details filled out, you can then choose what Events should create Notifications. They are pretty self-explanatory as to what triggers them, so we’re not going to break them down here. What we will talk about, as it will become important later, is the Event ID each of them has included in their Event Notification. They are, in order:

AirWatch Event Event ID
Device Enrollment 148
Device Unenrolled Enterprise Wipe 39
Device Wipe 25
Device Compromised Status Change 178
Device Compliance Status Change 184
Device Delete 662
Device Attribute Change No ID as it is not an actual event
Asset Number 641
Device Friendly Name 642
Organizational Group ID 218
User Email Address 643
Ownership 165
Operating System 163
Phone Number 645
Device MCC 646

There is a lot more information sent in the notifications, but the Event ID’s appear to be consistent across devices and OGs, so they should be something we can reliably look for to know what event has occurred.

Writing a Simple Python Script to Listen for HTTP Callbacks

As Python is the language I’m most familiar with at this point, and because my goal was to turn my existing scheduled scripts into event-driven scripts, I wanted a way to listen for the event notifications in Python. With some quick Googling, it looked like there were probably a few options, but I decided to go with Flask for this project.

So before getting started, and if it’s not already installed, run the following command to install Flask on your machine:

pip install flask

Now we can use Flask to listen for HTTP POSTs by running the following code:

#!/usr/bin/python

import json
from flask import Flask, request

app = Flask(__name__)

@app.route('/',methods=['POST'])

def main():
    data = json.loads(request.data)
    print data
    return "OK"

if __name__ == '__main__':
    app.run()

This code is quite simply listening (currently on your localhost or 127.0.0.1) for any HTTP POSTs. When one is received, it will parse the data passed into JSON, and then print that data out.

In order to have the same script listen on your host’s actual IP address, you can change the app.run() line to app.run(host='0.0.0.0'). It should be noted however that the code above requires no authentication to send data to it, so be cautious before just running this on an internet facing server or anything.

With AirWatch, this ends up spitting out some data like so:

{
  "EventId": 641,
  "EventType": "Asset Number",
  "DeviceId": 556,
  "DeviceFriendlyName": "My Device",
  "EnrollmentEmailAddress": "email@domain.com",
  "EnrollmentUserName": "username",
  "EventTime": "/Date(1525979516323)/",
  "EnrollmentStatus": "Enrolled",
  "CompromisedStatus": "",
  "CompromisedTimeStamp": "/Date(1525979516330)/",
  "ComplianceStatus": "Compliant",
  "PhoneNumber": "",
  "Udid": "3465FEB3BD615931A073832628A6D022",
  "SerialNumber": "C12345678910",
  "MACAddress": "012345678910",
  "DeviceIMEI": "3465FEB3-BD61-5931-A073-123456789",
  "EnrollmentUserId": 345,
  "AssetNumber": "09876543",
  "Platform": "AppleOsX",
  "OperatingSystem": "10.13.4",
  "Ownership": "CorporateDedicated",
  "SIMMCC": "",
  "CurrentMCC": "",
  "OrganizationGroupName": "Macs"
}

This output is from a notification about an Asset Number change. In a previous post I wrote about how we’re trying to automate creating Munki manifests with client details. One of the pieces of that script for us is to update the Asset Number in AirWatch with the asset number from our internal property database. We use this asset number for several different things, so it’s important that it be the actual asset tag as defined in the property DB. With the event notification, we can instantly see if an asset number was updated, and then quickly check to see if it matches the real asset number, and if not, change it to the proper number.

Side Note: We currently have set this up now in our environment, and while testing, within about 2 seconds of changing the asset tag to something incorrect, it is back to being the actual number again… it’s pretty amazing.

So let’s expand on our Python code above to add some logic that could catch an event like this, and then take the appropriate actions:

#!/usr/bin/python

import requests
import json
from flask import Flask, request

known_good_asset = '12345678'

def updateAssetTag(device_serial, good_asset_number):
    update_asset = requests.put('awmdm.host.com/api/mdm/devices?searchBy=serialNumber&id=%s' % device_serial, headers=request_headers, data={'AssetNumber':'%s' % good_asset_number})
    if update_asset.status_code == 204:
        print "Device Asset tag updated successfully for [%s]" % device_serial
    else:
        print "Unable to update device asset tag"

app = Flask(__name__)

@app.route('/',methods=['POST'])

def main():
    data = json.loads(request.data)
    try:
        ## We can pull out the key fields that matter to us from the event
        event_id = data['EventId']
        device_serial = data['SerialNumber']
    except KeyError:
        return "Data received was not in the format expected"

    if event_id == 641:
        ## We need to assign the device_asset here as opposed to above,
        ## because the AssetNumber is not passed with all event notifications
        device_asset = data['AssetNumber']
        if not device_asset == known_good_asset:
            updateAssetTag(device_serial, known_good_asset)

    return "OK"

if __name__ == '__main__':
    app.run()

This is clearly a bit of pseudo-code as it would always set the asset to a defined string for every single device that had an asset number change, but hopefully the logic is clear.

While a fairly simple example, it 1) shows just how quickly you can react to events taking place in your environment, and 2) is far less intensive than iterating over 1000 machines, checking the asset number for each device, updating the asset if one is out of sync, all while doing this on a schedule over and over again. Using Event Notifications, we can see a change for a specific device, look up the info just for that device, make the change for just that device, and then go back and quietly wait for the next event notification.

We are already beginning to build better workflows around these Event Notifications, such as using AirWatch as our authority on device status. Meaning, if a device is removed from AirWatch, we want to remove it from all of our other systems (i.e. Munki, MunkiReport, Chef, etc.). With the “Device Deleted” event, we can monitor for the removal of devices, and then instantly remove that device from all other systems to ensure we are not holding onto crufty data.

We are also looking at possibly using the notifications to allow for more granularity, and more options, for things like AirWatch’s compliance policies. At the moment, AirWatch has fairly limited capabilities when a device becomes Non-Compliant. But with event notifications, we could write the code to do whatever we wanted, whether that be to trigger an action, move the device to a more locked down OG, send an Install Application command, etc. The options become far greater and allow you to grow your environment beyond what a product may offer out of the box!

As always, thanks for reading and happy automating!

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

w

Connecting to %s