Monday, July 31, 2017

Isolating Citrix Cloud in your Azure Tenant

I have recently been studying issues that customers are having when trying to stand up a proof-of-concept environment for Citrix Cloud in Azure.

Most of these customers are standing up the full XenApp and XenDesktop Service.  However, our Citrix Cloud Services all have the same basic needs for any customer:
  1. Azure Subscription (for workers and infrastructure)
  2. App Registration (this is an Azure Tenant service account for our cloud based control plane to perform worker lifecycle events within a subscription)
  3. Virtual Network (the machines need IP addresses)
  4. Active Directory (there is a much larger discussion here, but either a read / write Domain Controller VM  or the Azure Active Directory Domain Service will work)
  5. The DNS setting for the Virtual Network must be your Active Directory 
  6. Cloud Connector machines (the connection between the machines in the subscription and the control plane)
  7. Some type of 'golden' image that is provisioned into the worker machines your end customers get their work done on.

Growing this conversation from the bottom up;

Each customer of Azure has at least one Azure Tenant.
This is your account in Azure.  It is the highest level of connection between Azure and you the customer.
Within your Azure Tenant you have Subscriptions.
Subscriptions are billing boundaries and service boundaries (services within subscription cannot 'talk' to each other without extra work, as if they are in different buildings).

Isolating Citrix Cloud in your tenant;


Can you isolate Citrix Cloud to its own Subscription in your Azure Tenant?  Yes!  And that is actually the topology that I am going to describe here.  How to isolate Citrix Cloud from your corporate infrastructure.

Common project slow down points that I have heard are:  modifications to existing virtual networks and protecting Active Directory.  

Focusing on the Virtual Network issue first;

You CAN create a virtual network dedicated to your Citrix Cloud deployment. 
The important things to remember are:
  • You need a route to your Active Directory
  • You must update the DNS settings of the Citrix Cloud virtual network to be the AD
The DNS setting is the most common place where customers trip up.  The DNS setting must be set. The Azure default results in the machines not being able to resolve the Active Directory.

The three models as pictures;

It is often that pictures tell a story faster and easier, I wanted to provide those to get you started thinking about your individual topology as well.

If your Active Directory is on the same Virtual Network you are most likely golden.

If your Active Directory machine(s) is on a different Virtual Network in the same subscription, you can use peering between the two virtual networks.

If your Active Directory machine(s) is on a different Virtual Network in a different subscription, you must use a gateway between the two virtual networks.

Friday, July 28, 2017

Virtual Network permissions for Citrix Cloud

In a previous post I covered how to manually create a Service Principal (App registration) for XenDesktop Essentials.  (this also applies to the XenApp and XenDesktop Service)

If you recall, this is the identity that Citrix Cloud will be using when it performs machine lifecycle actions in your Azure Subscription.

Things with permissions can get a bit strange in Azure pretty quickly.  One such area is Virtual Networks.

First of all, a Virtual Network exists within a Subscription.  It can belong to any Resource Group for management, but can be used by any machines or services within the subscription.

Now, in the world of assumptions, this is all fine and easy if you grant the Service Principal account the Contributor role AND the resource group that your virtual network belongs to is within a resource group under that same subscription.  You can take advantage of the inheritance.

This is not always the case.  In fact, it might not be the case for you at all.  You might be putting very tight controls on that Virtual Network to ensure it never gets messed up.

The minimum permissions that the Service Principal needs to your Virtual Network is the VM Contributor Role.  This level of access is necessary for the automated provisioning and lifecycle of desktop or session workers.

If you have a need to grant access to your Virtual Network or want to constrain access to your virtual network, here is how.

Remove the inheritance at the Virtual Network Resource Group from the subscription if it is enabled.
Explicitly grant the App Registration the VM Contributor role on the Virtual Network where worker machines will be attached when provisioned.

You can find more about the permissions in this article that I authored:  Manually granting Citrix Cloud Access to your Azure Subscription

Thursday, July 27, 2017

Azure Resource Manager Templates for Citrix Cloud workloads

At Citrix we recognize that different customers need different tools to accomplish their goals.  In the end, it is all about selecting the right tools for your environment and business processes to get you moving forward in an efficient way.

It has been brought to our attention that getting started in Azure with Citrix Cloud is not necessarily as straightforward as it needs to be, especially when customers go it alone (without the aid of a sales engineer or an integrator).

You will be seeing different tools, recommendations, updated documentation, and product enhancements to help get you (the customer) moving forward with your demonstration project, that Proof-of-Concept project, and moving into full production.

One of those tools was recently mentioned on this blog: Citrix Cloud XenDesktop Resource Location ARM Template

Without modification of the template this Azure Resource Manager template is focused on getting you up an going with that very first Demo environment.
It provides everything from an Active Directory Domain to NetScaler VPX.  And the glue in between to make it all work.

Additionally, there are other Azure Resource Manager templates that are componentized to support you in building out the infrastructure in your own way or integrating with your current Azure environment for any of the Citrix Cloud offerings.

These are being built to bring success to your Proof-of-Concept and production deployments. You can find the PoC and Production template repository here: https://github.com/citrix/CitrixCloud-ARMTemplates

This is a community repository and we would love to see your additions and suggestions.

I would also like to hear your stories and questions about using Azure to deploy your Citrix Cloud service, whether it be XenApp Essentials, XenDesktop Essentials, or XenApp and XenDesktop Service. 

Lets make it better together.


Friday, April 14, 2017

Getting started with Citrix Essentials on Azure


Earlier this month two Citrix Essentials products hit the Azure Marketplace;
XenApp Essentials and XenDesktop Essentials. https://www.citrix.com/blogs/2017/04/03/xendesktop-essentials-xenapp-essentials-now-available-in-azure-marketplace/

In this short period of time, there has been  customers who have purchased the services or are kicking the tires.
While I didn't give a number, I can say that it has been a pretty exciting first two weeks, and the interest from customers has been great.  Really great.

Both Essentials offerings run on Azure (exclusively) and are managed through Citrix Cloud.

Since these are new services, the documentation is constantly coming on-line. Here are some references that should get you over the initial hurdles of understanding how to implement it all.

The newly updated XenDesktop Essentials guide: http://docs.citrix.com/en-us/citrix-cloud/xenapp-and-xendesktop-service/xendesktop-essentials.html

If you were wondering if you could take advantage of Azure Active Directory Domain Service? Yes, you can: https://www.citrix.com/blogs/2017/04/11/xenapp-xendesktop-services-support-azure-ad-domain-services/

If you are in a hybrid cloud scenario (user workers in Azure with a VPN back to the datacenter where they need Kerberos or Windows pass-through authentication) you will need to setup up an Active Directory replica server in Azure: https://docs.microsoft.com/en-us/azure/active-directory/active-directory-install-replica-active-directory-domain-controller

Are you setting up Windows Desktops in Azure? 
Be aware that you must implement Azure Active Directory. More details on the way.

And take advantage of the Hybrid Use Benefit Windows 10 image in the Azure Gallery if you create a new golden desktop image.

Friday, April 7, 2017

Active Directory with XenDesktop Essentials in Azure

XenDesktop Essentials and XenApp Essentials have hit the Azure Marketplace, and they are catching on.

For those of you that remember Azure RemoteApp, XenApp Essentials is the replacement for that.  And for those of you that want to give Windows Client desktops to your user-base, XenDesktop Essentials is for that.
And, for those that want it all, There is XenApp and XenDesktop Service.  Which offers it all.

Now, the reason for my post.  Active Directory and Azure Active Directory.
There is a requirement of all of these solutions that the provisioned machines are joined to a domain.  This is where I see many folks getting confused between all of the various Active Directory options.

In reality, there are only two models that will work today (at the date of this post).  Let me describe them in terms of what you need to accomplish.

In both models, you have the user side running in Azure.  Whether that be XenApp Servers (Terminal Servers for you really old folks) or Desktops (Windows Client or Windows Server desktops).

Your answer to this next question defines the path that you need to head down.

Do your Azure based user sessions need to access resources in some other cloud / datacenter?
A different way to ask this - do you need a VPN between your users in Azure and whatever other resources they need to access in some other cloud / datacenter.

If your answer was no
Then I am calling you 'cloud born' or 'Azure based'.
Knowing this you can use Azure AD plus Azure Active Directory Domain Service.

AD Sync is built in, and most likely Azure AD is your source for users.  But you need the additional service to support domain join, group policy, and those traditional things that Active Directory provides.

I personally love the following guide for getting AADDS all up and running: Azure Active Directory Domain Services for Beginners

The trick here is that you need to use FQDNs for domain joins and domain references.  If you customize your Azure AD domain, use that.  If you don't it is YourDomain.onmicrosoft.com.

When you need to add Group Policy to lock things down; https://docs.microsoft.com/en-us/azure/active-directory-domain-services/active-directory-ds-admin-guide-administer-group-policy

If your answer was yes;
Then you are more of a 'traditional' enterprise that is in some hybrid deployment model.
Knowing this you need to use Azure AD plus Active Directory.

You will need to enable AD Sync, you will need to establish a replica domain controller in Azure, and you (probably) already have a VPN between your datacenter and Azure virtual network.

The replica domain controller in Azure: https://docs.microsoft.com/en-us/azure/active-directory/active-directory-install-replica-active-directory-domain-controller
Active Directory Sync / Connect to Azure AD: https://docs.microsoft.com/en-us/azure/active-directory/connect/active-directory-aadconnect
(It does not matter where you install / run that, just that you do).

In both cases; Don't forget to update the DNS settings of your Virtual Network with these new machine IP addresses.


Wednesday, April 5, 2017

Manually creating a Service Principal for XenDesktop Express

I have been looking at the customer experience around XenDesktop Express lately, and I have helped a few customers with issues around defining their Service Principal accounts.

Backing up a bit.  What is this 'Service Principal' account and what is it used for?

The Service Principal is the username / secret that is used by Citrix Cloud to talk to the Azure API and perform machine lifecycle actions in your Azure Subscription.

You could call it a delegated user, or an application user, or simply an application account.
The Service Principal is not a new concept in the enterprise world.  In my background we always created very restricted user accounts for use by applications, granting only those permissions that were necessary for the application to perform its functions.

I know there is guidance on using various PowerShell scripts to do this.  But quite honestly, it is so few clicks in the Azure Portal, you might as well do it there.  Far less hassle than installing the Azure cmdlets.

Plus - by doing it this way, you can quickly identify if you have the permissions necessary, and get it fixed or pass the responsibility to the person that can do it.

First, login to the Azure Account that 'Citrix' will be deploying workstations to.
Next make sure that you have a subscription container for the 'Citrix stuff' and a Virtual Network for the workstations to use all ready to go.


Create the App Registration / Service Principal
  1. Select the Azure Active Directory blade in the Azure Account
  2. Select 'App registrations'
  3. Select 'Add +'
  4. Enter a name, leave the application type as web app / API, and enter a Sign-on URL such as 'https://localhost/xde'
  5. Select Create
Grant it permission to interact with the Azure API for your account
  1. Once the registration is created, select it to view its settings
  2. Select 'Required permissions'
  3. Select 'Windows Azure Active Directory'
  4. Select 'Sign in and read user profile' and
  5. Select 'Read all users' basic profiles'
  6. Select 'Save'
  7. Select Add, Select an API, Select 'Windows Azure Service Management API', Select 'Select'
  8. Select 'Access Azure Service Management as organization users'
  9. Select 'Select'
  10. Select 'Done'
Add a Key (the secret)
  1. In the Settings, Select 'Keys'
  2. Enter a Key description, select a duration
  3. Select 'Save'
  4. Copy the Value of the key  (this value is necessary when this Service Principal is used with Citrix Cloud - and there are warnings that you can never see this key again)
Grant the Service Principal access to the Subscription for 'Citrix stuff'
  1. Select the Billing Blade
  2. Select the Subscription that you would like Citrix Cloud to be using
  3. Select 'Access control'
  4. Select '+ Add'
  5. Under 'Role' select 'Contributor'
  6. Under Select, type in the name of the App Registration you created (mine was 'xendesktop')
  7. Select the Azure AD user
  8. Select 'Save'
At this point in time, the Service Principal information can be handed off to your Citrix Administrator for establishing the Host connection to Azure in the Citrix Cloud portal.  
When Adding the Connection select the 'Use existing' option.

They will need;
  • the Subscription UUID
  • the Active Directory ID
  • the Application ID
  • the Application secret (that value that I mentioned you had to copy and save)
If you return the Azure Active Directory blade, Select the Properties, you will find the Directory ID.
Then select App registrations, select the one you created you can find the Application ID.
The Subscription id, is back under the Billing blade.


Tuesday, February 7, 2017

A reason to use state with Octoblu

I have been posting an 8 part series going though some advanced use of Octoblu.

Part 1: Use configuration events in Octoblu
Part 2: Creating custom devices in Octoblu
Part 3: Setting the state of an Octoblu device from a flow
Part 4: Listening to and acting on device state change in Octoblu
Part 5: Breaking values into new keys with a function node
Part 6: Reformatting nested JSON with JavaScript
Part 7: Logical data nesting with your Octoblu state device
Part 8:

Back at the beginning I introduced the concept of a state device.

Now, if you aren't yet understanding why I might introduce a state device, consider this:


Have you ever found yourself using SetKey and GetKey within flows to persist data, even if only for a little while?

Have you ever run into complex timing issues where you would love to break something into multiple flows and instead end up with one huge complex one?


This is where the state device is an easy fit.  Persist your data, in a common object that you can reference between flows.

Then, instead of relying on some message chugging through the system, you act upon a change to your state device.  So you could dev null some message, perform no update and exit your logic stream.

In the example I have been laying out I have two primary scenarios: 

Scenario 1: there are multiple incoming data sources
I have multiple devices that are all similar and they are feeding in data that I need to evaluate in a common way.  Each flow can update my state device independently, and then I simply have one evaluation flow to determine if I am going to send out my alert.

Scenario 2: there are multiple data listener paths
Just the opposite.  I have one primary input data source, it is big and complex.
Then I have multiple flows, each of which evaluates a specific type of data or specific properties.

Either way, it allows me to compartmentalize my flow logic and reduce / remove redundancy across the system.

So I end up with something like this:
Combined with this:
To do what I was doing in the first screenshot.

The big upside for me is that I removed all of the hardcoded naming filtering that I started with in order to persist the data.
The flows are now able to be more dynamic and handle the same sets of data no matter if it was mine, or someone else's.






Monday, February 6, 2017

Referencing nested array values in JavaScript from my Octoblu state device

Part 1: Use configuration events in Octoblu
Part 2: Creating custom devices in Octoblu
Part 3: Setting the state of an Octoblu device from a flow
Part 4: Listening to and acting on device state change in Octoblu
Part 5: Breaking values into new keys with a function node
Part 6: Reformatting nested JSON with JavaScript
Part 7: Logical data nesting with your Octoblu state device

Okay, here is the big post that I have spent an entire week working up to.

I have to admit, I don't write code every day and am self taught JavaScript (along with Python,  PowerShell, and batch) so this took me a while to work through.

From my last post, my incoming message looks this this:

{
  "msg": {
    "rooms": {
      "Redmond": {
        "lunch": {
          "motion": {
            "name": "Redmond_lunch_motion",
            "mapTitle": "Redmond",
            "room": "lunch",
            "device": "motion",
          },
          "refrigerator": {
            "name": "Redmond_lunch_refrigerator",
            "mapTitle": "Redmond",
            "room": "lunch",
            "device": "refrigerator",
          },
          "door": {
            "name": "Redmond_lunch_door",
            "mapTitle": "Redmond",
            "room": "lunch",
            "device": "door",
          }
        }
      }
    },
    "fromUuid": "d5b77d9b-aaf3-f089a7096ee0"
  },
  "node": "b5149300-9cbd-1f1b56e5d7bb"
}

There can be a variable number of devices per room, and a variable number of rooms per map, and a variable number of maps.  My nesting pattern above is rooms.map.room.devices

Now for the hard part.
I want to evaluate the differences between values of different devices, per room.
This ends up being a lesson in how values are referenced in arrays in JavaScript.

Before I move forward, I have abbreviated the above JSON to spare you scrolling.  There are additional fields, and these additional fields contain Date objects that I am interested in.  And these Dates are formatted as odd numbers which are actually Epoch Time.

So, to give you the full treatment, here is a real message:

{
  "msg": {
    "rooms": {
      "Redmond": {
        "lunch": {
          "motion": {
            "name": "Redmond_lunch_motion",
            "mapTitle": "Redmond",
            "room": "lunch",
            "device": "motion",
            "motion": false,
            "motion_updated_at": 1485539337.480442,
            "battery": 1,
            "battery_updated_at": 1485539337.480442,
            "tamper_detected": null,
            "tamper_detected_updated_at": null,
            "temperature": 21.666666666666668,
            "temperature_updated_at": 1485539337.480442,
            "motion_true": "N/A",
            "motion_true_updated_at": 1485539125.483463,
            "tamper_detected_true": null,
            "tamper_detected_true_updated_at": null,
            "connection": true,
            "connection_updated_at": 1485539337.480442,
            "agent_session_id": null,
            "agent_session_id_updated_at": null,
            "connection_changed_at": 1485175984.3230183,
            "motion_changed_at": 1485539337.480442,
            "motion_true_changed_at": 1485539125.483463,
            "temperature_changed_at": 1485529054.5705206
          },
          "refrigerator": {
            "name": "Redmond_lunch_refrigerator",
            "mapTitle": "Redmond",
            "room": "lunch",
            "device": "refrigerator",
            "opened": false,
            "opened_updated_at": 1485539969.6240845,
            "tamper_detected": false,
            "tamper_detected_updated_at": 1476739884.682764,
            "battery": 1,
            "battery_updated_at": 1485539969.6240845,
            "tamper_detected_true": "N/A",
            "tamper_detected_true_updated_at": 1476739866.2962902,
            "connection": true,
            "connection_updated_at": 1485539969.6240845,
            "agent_session_id": null,
            "agent_session_id_updated_at": null,
            "opened_changed_at": 1485539969.6240845
          },
          "door": {
            "name": "Redmond_lunch_door",
            "mapTitle": "Redmond",
            "room": "lunch",
            "device": "door",
            "opened": false,
            "opened_updated_at": 1485538007.9089093,
            "tamper_detected": null,
            "tamper_detected_updated_at": null,
            "battery": 1,
            "battery_updated_at": 1485538007.9089093,
            "tamper_detected_true": null,
            "tamper_detected_true_updated_at": null,
            "connection": true,
            "connection_updated_at": 1485538007.9089093,
            "agent_session_id": null,
            "agent_session_id_updated_at": null,
            "opened_changed_at": 1485538007.9089093
          }
        }
      }
    },
    "fromUuid": "d5b77d9b-aaf3-f089a7096ee0"
  },
  "node": "b5149300-9cbd-1f1b56e5d7bb"
}

Now, the output I am looking for is to take some of these sensor Date values and evaluate them between each of the three devices.  Such as: door-refrigerator, motion-door, motion-refrigerator and so on.

If these values were in the same part of the message, it would be really easy.  I could simply dot reverence the values and do the math.
But they are not.  Each sensor is in its own document, in an array.

Now, if you recall a few posts back, I have a naming convention and I am standardizing three of the names:  "door", "refrigerator", and "motion".  Those I am not allowing to change.  But the room can and the map can.

Recall, I began this exercise this with just an array of devices with values.  Processed them to group by a logical naming pattern, saved that to an Octoblu state device, and now I am further processing that into my actionable data which I can easily handle with Octoblu filters to handle alerting or whatever I want to do.

So, to get you to read to the end and not just steal my code here is the output that I am producing, per room.
This gives me a nice single document per room as output - I can pass that to a demultiplex node to break the rot array apart and evaluate each document. 

My output looks like this:

{
  "msg": [
    {
      "motion": "motion",
      "motionAt": 1485544607.3195794,
      "motionAtHuman": "2017-01-27T19:16:47.319Z",
      "mapTitle": "Redmond",
      "room": "lunch",
      "refrigerator": "refrigerator",
      "fridgeOpenedAt": 1485539969.6240845,
      "fridgeOpenedAtHuman": "2017-01-27T17:59:29.624Z",
      "door": "door",
      "doorOpenedAt": 1485538007.9089093,
      "doorOpenedAtHuman": "2017-01-27T17:26:47.908Z",
      "diffDoorsOpenedMinutes": 32,
      "diffDoorMotionMinutes": 109,
      "diffRefrigeratorMotionMinutes": 77,
      "sinceDoorOpenMinutes": 115,
      "sinceRefrigeratorOpenMinutes": 82,
      "sinceMotionMinutes": 5
    }
  ],
  "node": "98cb8680-a264-1b8483214e06"
}

Now, to end this long, long story the JavaScript is below.
What I tried to do was have an intuitive way to read the code and reference each level of the document arrays, so you could understand where you were in the hierarchy.

// array to output
var output = [];
for ( var map in (msg.rooms) ){
    for ( var room in msg.rooms[map] ){
        var doorOpenedAt;
        var fridgeOpenedAt;
        var motionAt;
        var roomOutput = {};
        for ( var sensor in msg.rooms[map][room]){
            switch ( msg.rooms[map][room][sensor].device ) {
                case "door":
                    doorOpenedAt = moment.unix(msg.rooms[map][room][sensor].opened_changed_at);
                    roomOutput.door = msg.rooms[map][room][sensor].device;
                    roomOutput.doorOpenedAt = msg.rooms[map][room][sensor].opened_changed_at;
                    roomOutput.doorOpenedAtHuman = doorOpenedAt;
                    break;
                case "refrigerator":
                    fridgeOpenedAt = moment.unix(msg.rooms[map][room][sensor].opened_changed_at);
                    roomOutput.refrigerator = msg.rooms[map][room][sensor].device;
                    roomOutput.fridgeOpenedAt = msg.rooms[map][room][sensor].opened_changed_at;
                    roomOutput.fridgeOpenedAtHuman = fridgeOpenedAt;
                    break;
                case "motion":
                    motionAt = moment.unix(msg.rooms[map][room][sensor].motion_true_changed_at);
                    roomOutput.motion = msg.rooms[map][room][sensor].device;
                    roomOutput.motionAt = msg.rooms[map][room][sensor].motion_true_changed_at;
                    roomOutput.motionAtHuman = motionAt;
                    break;
            } // close of switch
            roomOutput.mapTitle = msg.rooms[map][room][sensor].mapTitle;
            roomOutput.room = msg.rooms[map][room][sensor].room;
        }  // close of sensor
        roomOutput.diffDoorsOpenedMinutes = Math.abs(doorOpenedAt.diff(fridgeOpenedAt, 'minutes'));  //removing Math.abs will give a + - if the refrigerator opens and the door does not it will be negative
        roomOutput.diffDoorMotionMinutes = Math.abs(doorOpenedAt.diff(motionAt, 'minutes'));
        roomOutput.diffRefrigeratorMotionMinutes = Math.abs(fridgeOpenedAt.diff(motionAt, 'minutes'));
        roomOutput.sinceDoorOpenMinutes = moment().diff(doorOpenedAt, 'minutes');
        roomOutput.sinceRefrigeratorOpenMinutes = moment().diff(fridgeOpenedAt, 'minutes');
        roomOutput.sinceMotionMinutes = moment().diff(motionAt, 'minutes');       
        output.push(roomOutput);
    }  //close of room
} // close of map
return output;


Lots of leading up to this post.  But I like to expand folks' understanding along the way.
And I know we don't all tolerate long articles.

I can thank Tobias Kreidl for even getting me started on this series of posts.
He asked a simple question, and I had a final answer, but I wanted to tell the journey so that he understood how I got to where I did.
That leaves it up to you to take what you need.  That's just how I write and respond to questions.

Friday, February 3, 2017

Logical data nesting with your Octoblu state device

Part 1: Use configuration events in Octoblu
Part 2: Creating custom devices in Octoblu
Part 3: Setting the state of an Octoblu device from a flow
Part 4: Listening to and acting on device state change in Octoblu
Part 5: Breaking values into new keys with a function node
Part 6: Reformatting nested JSON with JavaScript

In my last post I left you with some JavaScript to reformat a JSON message and come out with a nice new format.

I left you hanging with my key format though.
Why did I format my key names the way I did?

It is actually pretty simple in concept (but it took me a long time to get all the code right).

Previously I mentioned that after I $set the data on my Octoblu state device, I want to catch that data change in another workflow.
And I also mentioned that logically grouping that data would make it easier to visualize and work with farther down the chain.

So, back to the key name pattern in my output:

{
  "msg": {
   "rooms.Redmond.lunch.motion": {
     "name": "Redmond_lunch_motion",
     "mapTitle": "Redmond",
     "room": "lunch",
     "device": "motion",
   },
   "rooms.Redmond.lunch.refrigerator": {
     "name": "Redmond_lunch_refrigerator",
     "mapTitle": "Redmond",
     "room": "lunch",
     "device": "refrigerator",
   },
   "rooms.Redmond.lunch.door": {
     "name": "Redmond_lunch_door",
     "mapTitle": "Redmond",
     "room": "lunch",
     "device": "door",
    }
  },
  "node": "e271a6c0-9f9b-8d7882b7836a"
}


This is all about the $set.
I want each set of devices grouped under their room which is under the map they correspond to.
Thus: rooms.map.roomName.deviceName

Which is passed to a JSON Template Node in Octoblu which very simply looks like this:

{
 "$set": {{msg}}
}


Here is where there are different patterns for referencing the message values in Octoblu.
If you are referencing a blob don't put quotes around the mustache notation like I did above.
If you are referencing a value, then put double quotes around the value like this:  "rooms.{{msg.name}}"

The hard thing to get right is the quotes.  Since you will get a false message that your JSON is improperly formatted from the editor, when the message that comes out is actually totally right.

Now, back to why I had the dot notation key name.

When I listen to my state device for a change I will get this nice hierarchy as the output.  And I persist my data nice and logically.

{
  "msg": {
    "rooms": {
      "Redmond": {
        "lunch": {
          "motion": {
            "name": "Redmond_lunch_motion",
            "mapTitle": "Redmond",
            "room": "lunch",
            "device": "motion",
          },
          "refrigerator": {
            "name": "Redmond_lunch_refrigerator",
            "mapTitle": "Redmond",
            "room": "lunch",
            "device": "refrigerator",
          },
          "door": {
            "name": "Redmond_lunch_door",
            "mapTitle": "Redmond",
            "room": "lunch",
            "device": "door",
          }
        }
      }
    },
    "fromUuid": "d5b77d9b-aaf3-f089a7096ee0"
  },
  "node": "b5149300-9cbd-1f1b56e5d7bb"
}


Next post:  I am going to parse all that nesting back apart and make yet another message


Thursday, February 2, 2017

Reformatting nested JSON with JavaScript

Part 1: Use configuration events in Octoblu
Part 2: Creating custom devices in Octoblu
Part 3: Setting the state of an Octoblu device from a flow
Part 4: Listening to and acting on device state change in Octoblu
Part 5: Breaking values into new keys with a function node

In my last example for an Octoblu Function node I very simply took the string value from a key and using .split() broke it into new fields.

This had the dependency of a format convention for the string.

Now, what if I had an incoming array which contained that data. 
And I only wanted to select certain values within each array element, and I had additional data which was nested within another array that I want to bring up a level to make it easier to evaluate later on.

To describe this differently, lets look at (an abbreviated version of) my incoming message:

{
  "msg": {
    "data": [
      {
        "uuid": "3b8a7529-b0f1-ddba9dc4cc27",
        "desired_state": {
          "pairing_mode": null
        },
        "last_reading": {
          "connection": true,
          "connection_updated_at": 1480529975.6920671,
        },
        "hub_id": "509234",
        "name": "Redmond",
        "locale": "en_us",
        "units": {},
        "created_at": 1476738922,
        "triggers": []
      },
      {
        "last_event": {
          "vibration_occurred_at": null
        },
        "uuid": "4d60c8ad-b6d2-f17c5e4a1192",
        "desired_state": {},
        "last_reading": {
          "motion_changed_at": 1480490895.7546704,
          "motion_true_changed_at": 1480490698.2846074,
          "temperature_changed_at": 1480530247.413451,
          "connection_changed_at": 1480530247.413451
        },
        "name": "Redmond_lunch_door",
        "triggers": []
      },
  },
  "node": "5c0e3d40-bddd-6f97ce016844"
}


I have an incoming message (msg) it has an array (data) of documents.  The data within each document could be different as each is a different device with different capabilities and settings.

From this point I have a couple wants: I need the name information of the sensors (from my previous post), and I need to in-nest the values of last_reading to make it easier to handle down the line.

And, then I want to save this information to my Octoblu device (a few blog posts ago).

Lets just format on the array at this point, I don't want this to get too confusing.

//A document object to hold the sensors per room
var rooms = {};


for ( var i in (msg.data) ){



 var sensor = {}; //an empty document object to populate with new key:values



 sensor.name = msg.data[i].name; //the incoming name

 var dotName = (msg.data[i].name).replace(/_/g,"."); 

   //the name in dot notation instead of underscores (see the next post)


 // break the device name into its descriptors (from the last post)
 var descriptors = (msg.data[i].name).split('_');

 switch(descriptors.length){
  case 3:
   sensor.mapTitle = descriptors[0];
   sensor.room = descriptors[1];
   sensor.device = descriptors[2];
   break;
  case 2:
   sensor.mapTitle = descriptors[0];
   sensor.device = descriptors[1];
   break;
  case 1:
   sensor.device = descriptors[0];
   break;
 }

 // un-nest last_reading to make it easier to handle later on
 var last_reading = msg.data[i].last_reading;
 for ( var reading in last_reading ){
  sensor[reading] = last_reading[reading];
 }

 // only those devices with a room value
 if ( sensor.room ) {
  var room = {};
  dotName = "rooms." + dotName;
  rooms[dotName] = sensor;

  // in the end, I want the devices of a room under the key pattern for that room
 }
}
return rooms;


This is what I get back out:

{
  "msg": {
   "rooms.Redmond.lunch.motion": {
     "name": "Redmond_lunch_motion",
     "mapTitle": "Redmond",
     "room": "lunch",
     "device": "motion",
     "motion": false,
     "motion_updated_at": 1485539337.480442,
     "connection_changed_at": 1485175984.3230183,
     "motion_changed_at": 1485539337.480442,
     "motion_true_changed_at": 1485539125.483463,
     "temperature_changed_at": 1485529054.5705206
   },
   "rooms.Redmond.lunch.refrigerator": {
     "name": "Redmond_lunch_refrigerator",
     "mapTitle": "Redmond",
     "room": "lunch",
     "device": "refrigerator",
     "opened": false,
     "opened_updated_at": 1485539969.6240845,
     "connection_updated_at": 1485539969.6240845,
     "opened_changed_at": 1485539969.6240845
   },
   "rooms.Redmond.lunch.door": {
     "name": "Redmond_lunch_door",
     "mapTitle": "Redmond",
     "room": "lunch",
     "device": "door",
     "opened": false,
     "opened_updated_at": 1485538007.9089093,
     "connection_updated_at": 1485538007.9089093,
     "opened_changed_at": 1485538007.9089093
    }
  },
  "node": "e271a6c0-9f9b-8d7882b7836a"
}


Next post:  How that dot notation key name pattern is useful to me.

Wednesday, February 1, 2017

Breaking a value into new keys with a function node

Part 1: Use configuration events in Octoblu
Part 2: Creating custom devices in Octoblu
Part 3: Setting the state of an Octoblu device from a flow
Part 4: Listening to and acting on device state change in Octoblu

Previously I had set custom keys and then listened for settings changes.

What if I wanted to set an array of key:values, or a larger document.  How might I handle that with Octoblu?

You have two primary options:
  • collect nodes
  • F(x) (function nodes)
Collect Nodes emit when the collection hits its minimum value until they reset when it reaches its maximum value.  And what you get is a collection of what came in to that node.

The key here is that you need to predict or hard code when the size of the collection resets back to zero and it begins again.  That is not always easy.

So I frequently end up leaning on F(x) or Function Nodes - this is arbitrary, linear, JavaScript.  It can't loop, it can't wait.  You must construct it so that it executes as a very quick function taking the incoming message, running, and then outputting whatever you tell it to return.

By default a Function node gives you the line return msg;
Which would output exactly the message (msg) that came in.

If you ant to reference the value of a specific key you simply use dot notation such as return msg.rooms;

Everything you do here, requires that you understand the data in your incoming message.  And for longevity sake, that format does not change.

Many of the methods that you have in JavaScript are at your disposal.  The not so easy part is the debugging.  Because you don't get very good debug detail.  But with a bit of effort many folks can work through that.

Now, for my first example:  I am going to create new key:values from an existing value.

I have established a naming pattern of:  map_room_device

This allows me to name my devices in a structured way and then deal with them in Octoblu without using a large number of filters and hard coded values.  In essence, devices came come and go as long as I stick with my naming convention.

Now, I need to make this naming convention more useful and easier to work with farther down the line as the messages become properties, so I want to make new keys from the name.

var descriptors = (msg.name).split('_');


switch(descriptors.length){
 case 3:
  msg.mapTitle = descriptors[0];
  msg.room = descriptors[1];
  msg.device = descriptors[2];
  break;
 case 2:
  msg.mapTitle = descriptors[0];
  msg.device = descriptors[1];
  break;
 case 1:
  msg.device = descriptors[0];
  break;
}


return msg;


I always have error handling.  And that is why I have three cases.  In case I have a device that does not have a map or a room in the naming pattern.

The output of this is the addition of 1, 2, or 3 key:values to the outgoing message.

Just a simple case of what can be done.

Next, straight to a really big message and reformatting arrays..

Tuesday, January 31, 2017

Listening to and acting on device state change in Octoblu

Part 1: Use configuration events in Octoblu
Part 2: Creating custom devices in Octoblu
Part 3: Setting the state of an Octoblu device from a flow

Now we are on to Part 4 - Responding to state changes

If you are catching up, check out the previous posts.

In this post I am going to listen to state changes of an Octoblu device within a flow and then respond to that somehow.

I have my starter flow from last this that looks like this:


Now, I want to create a second flow, where I am going to simply wait for my key 'rooms' and then process that data.  In this flow I am listening to state changes to myDevice.

To begin with your new flow should look like this:


Turn on 'Use Configuration Events' but you don't need to turn on 'Use Incoming Messages' like you did in the first flow.  In this flow we are only listening, not sending messages to or modifying the properties of (like the previous flow).

Open a second browser window, open your previous flow and click the trigger.
Notice that in the debug of this flow, you get the same debug output.  Because you are listening for changes to the device (in both flows).

Now, add some operator after your device in the new flow, set it and modify your JSON in the first flow to send something you can begin to act on.  Use {{msg.rooms}} to reference my example value, but make your own, set multiples, have fun with it.

Quite honestly, it is that simple.
And what you have built is this special Thing, where you can now save JSON formatted data, and then catch when it changes in some other flow.

one to one, many to one, one to many.....
And this data is all yours, formatted by you.

Next up, some of the screwy ways I have dealt with JSON data.


Monday, January 30, 2017

Setting the state of an Octoblu device from a flow

Building on what I began:

Part 1: Use configuration events in Octoblu
Part 2: Creating custom devices in Octoblu

In this post I am going to set and unset properties and their values on an Octoblu Thing, from a flow.

I am going to use the custom Thing that I created in my previous post, but you can use any Thing that has the option "Use Configuration Events".

Part 3 - Setting properties dynamically from a flow

In the previous post I created a custom Thing. And named it: MyDevice

Now I create a new flow, and I add that Thing to the Flow.


I select the Node (a reference to that thing in this flow) and turn on Use Configuration Events.  Notice the little gear.  This means that the behavior of this node in this flow is now different.

The next advanced thing that I am going to do is to add Trigger and JSON template nodes to define the message that is being sent and I am going to turn on Use Incoming Message on my custom node.

Let me back up here and explain a little..
Use Incoming Message takes whatever message is sent to the node.  If you don't turn this on, your device must have fields that you can set and then you can reference the values of keys using mustache notation or hard code values.

And then I am going to attach a debug node after my Thing.


And this makes a complete message circuit from start to end.

Be careful to pay attention to what you are doing - DO NOT create loops; they are very, very bad.  You will get your account suspended.

Now, to craft some JSON.

This is a very simple JSON body that we put in the template:
{
 "$set": {
  "rooms": "foo"
 }
}

Set the value for the key rooms to foo.

Start the flow, click the Trigger, and look for the Key 'rooms' in the message output.
In fact, explore that message output a bit.  Notice, these are all the settings / properties / state values of your device.

Now, change the JSON.  Change the key name, or change the value and see what happens.

If you want to remove the key and its values, we unset.
{
 "$unset": {
  "rooms": ""
 }
}

The $set and $unset are actually MongoDB commands.  And you can use others, such as $addToSet, $push, etc.  As long as you format your JSON properly.

Now, I have discovered that there are some key names that must be special to Octoblu and if you try to use them, nothing happens.  I am going to mention that in case you run into doing everything right, but nothing changes.
Is there a list of these special keys?  Not that I know of.  I was simply observing behavior...

Next up:  responding to state changes


Friday, January 27, 2017

Creating custom devices in Octoblu

Part 1: Use configuration events in Octoblu

This is something that I have been playing with for a very long time.
I directly use the Octoblu API to create my own devices.

I have copied existing devices, and I have created entirely new devices.
In this post I am going to create an entirely new device so that I can take advantage of the state (properties) feature to persist and evaluate data.

Why am I doing this instead of just reusing an existing Device?  Primarily because this makes a much easier to digest configuration change message on the listen side later on.  As there is a bare minimum number of properties.

Part 2 - Create a custom Octoblu device

Up front - I am a hard core Windows user (I do use Linux as well, but I have never used MAC) so my code example is in PowerShell.

Why would I create a custom device?
Well, my 'Thing' is an abstraction and not a physical device.

In practice, I have created 'rooms' in which I save data (aka set properties) from multiple sources or flows.  And I listen to specific data (property) changes of this device and then act on that data.

To do this there are a few things involved;
  • your security uuid and token
  • defining the security of the device
  • defining any custom properties of the device (if you want)
  • defining the base properties
  • performing the POST
  • looking at what you get back
Much of what is in the PowerShell below is all about building the object as a PowerShell object before it is converted to a JSON object.  Making sure that the data format is correct (arrays in the proper places, lists in the proper places, etc.)

# Octoblu User account (check your user properties)
$meAuthHeader = @{
    meshblu_auth_uuid = ''  
    meshblu_auth_token = ''
}


### build the JSON

# define the device permissions
$user = @()
$user += @{ uuid = $meAuthHeader.meshblu_auth_uuid }
$anyone = @()
$anyone += @{ uuid = '*' }
$empty = @()
$configureWhitelist = @{
    as = $empty;
    received = $empty;
    sent = $empty;
    update = $user
}
$message = @{
    as = $empty;
    received = $empty;
    sent = $anyone
}
$discover = @{
    as = $empty;
    view = $user
}
$broadcast = @{
    as = $empty;
    received = $anyone;
    sent = $anyone
}
$whitelists = @{
    configure = $configureWhitelist;
    message = $message;
    discover = $discover;
    broadcast = $broadcast
}

# define the schema of the device (behaviors and properties)

$properties = @{
    rooms = @{
        title = "sensors";
        type = "array";
        readonly = $true;
        items = @{
            type = "string"
        }
    }
}
$configure = @{
    Default = @{
        type = "object";
        properties = $properties;
        'x-form-schema' = @{
            angular = "configure.Default.angular"
        }
    }
}

$options = @{
    title = "Options";
    type = "object";
    properties = $properties
}
$configure = @{
    Default = @{
        title = "My Entity";
        type = "object";
        properties = @{
            options = $options
        };
        'x-form-schema' = @{
            angular = "configure.Default.angular"
        }
    }
}

$fields = @()
$fields += @{ key = "options.rooms" }
$form = @{
    configure = @{
        Default = @{
        angular = $fields
        }
    }
}

# wrap the above in the proper keys

$schemas = @{
    configure = $configure;
    form = $form;
    version = "2.0.0"
}

$meshblu = @{
    version = "2.0.0";
    whitelists = $whitelists;
}

# basic 'properties' (aka key:values) of the Device

$body = @{
    online = $true;
    owner = $meAuthHeader.meshblu_auth_uuid;
    type = "device:myDeviceType";
    name = "myDeviceName";
    city = "Redmond";
    meshblu = $meshblu;
    schemas = $schemas
}

# convert the PowerShell object to a JSON object

$json_body = $body | ConvertTo-Json -Depth 99

### create the device

$device = Invoke-RestMethod -URI http://meshblu-http.octoblu.com/devices -ContentType "application/json" -body $json_body -Method Post

$device | ConvertTo-Json -Depth 99


Now record the uuid and token of your device.  This is the ONLY time you get this token back, and you would need to generate a new one to know it again.

Also, you can set other properties such as a 'logo'.  For the logo the value must be an HTTPS URL reference to an SVG.

Next up:  Setting properties from a flow to this device.

Thursday, January 26, 2017

Use Configuration Events in Octoblu

It has been a really long time since I have written a post and I need to get back into practice.   So I thought I would share a few things that I have been doing with Octoblu for a few months now.

Part 1 - Use Configuration Events

First of all some background, if you are a user of Octoblu I am sure that you noticed a change a few months back to some of the Things.  A new option appeared: "Use Configuration Events"

This is the exposure of  device state for Octoblu Things.  In the software world 'state' is a highly overloaded term. 
As we look at the term state in relation to IoT devices there is a common pattern among most all the platforms - it really refers to a property or an abstract term to describe some setting change that might have a bunch of steps involved within it.
  • desired state  - this refers to IoT devices that can accept a setting and then do something to apply that setting.  The state we want this thing to be in.  You see this in many places from Desired State Configuration in Windows to the device twin model of Azure IoT Hub.
  • reported state - many folks consider this the true state.  The current condition that the system knows it is in.
In Octoblu - state is a property of a Thing or device.  For physical Things it works like this;
  • you set the state (define some key: value using JSON and apply that - it can be a known or unknown property).
  • the Connector that is running on the Thing then gets a notification that its state has been changed.
  • If the Connector knows how to interpret the setting it applies it.
  • The Connector then reports back.
The new Hue light connector is a good example of a state based Octoblu Thing.

You must turn on "Use Configuration Events" to change any setting (light on and color for example).

And then - if you attach a debug node to the right side of it, you can see the output of all of its properties.  More on this later as I expand on how I am using the device state capability in some of my work.