Monthly Archives: April 2014

VMware vCenter Operations data mining

*Warming – This is most likely totally unsupported and not how the product is intended to be used!

I’ve been working on a couple projects with vCOps recently (NO SPOILERS YET!) and came across something that was totally new to me that I wanted to share.   No internal secret sauce here either.  I found this via KB 2011714.

FYI – I’m not an expert on licensing, but I assume only the highest level license unlocks this along with the custom portal.

There is a way to query the vCOps system with SQL syntax from a web browser so you do not have to muck with any database connectors, hacking anything, or potentially breaking something.   Plug in the URL https://[vcops UI]/vcops-custom/dbAccessQuery.action    and you are greeted with a screen like the below.

I was looking for what the exact syntax was for host memory so I went through the tables one by one with select * statements until I found output that looked like it was what I recognized.   And BINGO!  I found it via syntax like this (heh, i said like there…get it?)

select * from attributekey where attr_key like ‘%mem%’ and attr_key like ‘%host%’

The metric I was looking for is  mem|host_usagePct

Screen Shot 2014-04-17 at 11.49.06 PM

Excellent but what can I do with that now?

vCOps 5.8, the most recent version at the time of this writing, has had an API of sorts for many versions.  I wouldn’t call it a REST API by any stretch but it does offer a way of getting data IN and OUT over HTTP.      (This is nothing ground breaking here, people have been talking about this at VMworld and blogs for a while now…)

How do I get data in?

On your vCOps system you can plug this URL in to get a definition of the API:  https://[vcops ui]/HttpPostAdapter/OpenAPIServlet


This interface is used to import metric data into vCenter Operations Manager, one resource at a time. This interface handles the creation of the resource and resource kind, as well as any new metric names. This is the default interface used when no “action” parameter is specified in the first line of body of the HTTP Post.

This is what we are looking for to add custom data.  Note it says no “action” parameter is specified.   Also note the unfamilar syntax for posting data.  You only provide the below URL, a POST method and this in the body.

first line
resourceName,adapterKindKey,resourceKindKey,identifiers,resourceDescription,monitoringInterval,storeOnly, sourceAdapter, disableResourceCreation

All subsequent lines
One metric per line with a comma-separated list of: metricName,alarmLevel,alarmMessage,date,value,thresholdHigh,thresholdLow

URL: https://[vcops ui]/HttpPostAdapter/OpenAPIServlet
MyData,Http Post,MyResource,,MyDescription,,yes,

Screen Shot 2014-04-18 at 12.01.35 AMAnd the value is added.   Note this is the time in MS not S.  Originally I was doing work from vCO and found the time in there was seconds and not milliseconds as vCOps expects, and….I was confused.  For a while.

Ok so you can add custom data so what…

The “So what” will be coming!  Be patient!

How do I pull data out?

Back to the original point of the post in figuring out that host metric syntax.     I learned from the API reference the  lookupResource action:


Use this interface to find existing resources matching specified resource name, adapter kind and resource kind.
Resource name matching can use string compare or regular expression matching. To enable regular expression matching specify “regex:” prefix followed by the matching pattern.
Adapter kind and resource kind parameters are optional. If not specified only name will be used for matching.
Return value is one row per found resource:
resourceName=[name of resource]&adapterKindKey=[adapterKindKey]&resourceKindKey=[resourceKindKey]&identifiers=[identifiers]

To use this interface, the body of the HTTP POST request should contain a single line:
action=lookupResource&resourceName=[name of resource]&adapterKindKey=[adapterKindKey]&resourceKindKey=[resourceKindKey]

This is useful in short, in case I know one of the values like the name of the resource (my host name), I can find the rest of the values for adapterKindKey,  resourceKindKey and Identifier.   This all probably makes sense to someone but it’s way confusing to me.  So here’s the shortcut:

Similar to the previous query, send this in the body.  Ah ha!



Screen Shot 2014-04-18 at 12.16.34 AM

Excellent!  But…Why?

Now we know all the variables needed to pull data!  If you check the API reference again you’ll find “ getMetricDataAndDT”


Use this interface to get collected data for a resource and a metric.
Results will be a CSV List (Including the header):

Time, Value, LowDT, HighDt, smooth
1365195172879, 37.5, 15.2, 73.3, 36.2 — with DT and Smooth values
1365195172879, 37.5, , , 36.2 — with no dt and smooth values
1365195172879, 37.5, , , — with no dt and no smooth

To use this interface, the body of the HTTP POST request should contain a single line:
action=getMetricDataAndDT&resourceName=[name of resource]&adapterKindKey=[adapterKindKey]&resourceKindKey=[resourceKind]&identifiers=[identifiers]&metricKey=[metrickey]&starttime=[startTime]&endtime=[endTime]&includeDt=[true|false]&includeSmooth=[true|false]

Now we know the resource name, adapter kind key, resource kind key, identifiers AND the metric key (which was the original point of this post!).  I love when a plan comes together…

Combine all the bits, and stick a start time in there (or just something really small to return everything… probably not recommended):


Screen Shot 2014-04-18 at 12.22.09 AM

And we have data!

Again..  The product is not really intended to be used this way, but all that data is in there.   Let’s do something special with it!

Tagged , , , ,

Blog As A Service (Application Director + WordPress + Twitter + Cyclocross)

It has been a while since I’ve posted a full project, so here it goes.  Enjoy.  This time we have our first guest appearance in a super awesome software developer, cycling, and beer snob buddy @jrrickard, who I am doing a number of projects with these days & showcasing some of them in a booth at our employers internal Science Fair (It’s going to be rad.  And there will be pliny.)    How did I get so far off track already…

The Story

I have been doing a lot with VMware vCloud Application Director lately, and while I think it is a really interesting product that strokes all of my ex-sysadmin nerdy tendancies in all the right ways, I think it has a really bright future as it’s a _really_ powerful tool that almost no one knows about (yet).  So, I thought to myself, what’s something totally silly I can do with it while still showing off it’s real world potential?  (if you haven’t picked up on it yet, that’s kind of my thing…)   That’s when I decided:


About ten minutes later, I have this:

Screen Shot 2014-04-14 at 11.57.58 PM


So what’s going on here?   Step by Step:

1) I tweet a specific phrase like “I should start a blog on cyclocross” .

2) @jrrickard wrote some slick python that talks to Twitter over the REST API, finds that phrase, and sends a specially formatted call over a REST API to vCO [1] [note that is supposed to be a superscript… not sure how to do that and link to a lower section in this blog yet..]

3) The vCO workflow(s) talk to AppD over a REST API, processes the parameters, finds the application blueprint in AppD, and schedules the deployment.  (There is some new fancy secret sauce another co-worker is developing that I may post about at a later time.  In short it is a collection of workflows that finds and executes exactly what you want in AppD based on the human readable name instead of the ID… which is a pain in the rear to find..  seriously…)

4) The AppD blueprint that gets deployed is based on a canned blueprint found at the VMware Solution Exchange here [2] with only a minor addition at the end.  It deploys a CentOS VM from a template, and installs Apache, MySQL and WordPress in order with all dependencies resolved.  AppD configures each one and starts all services.

5) Based on the original tweet that started this whole mess, an AppD service I created from scratch and added to the blueprint (a) pulls a CLI for WordPress from Git, (b) uses that cli to write a few posts to the blog on the topic I tweeted, and lastly (c) goes to flickr (over another REST API) and pulls a bunch of images on the topic to adds them to the posts.

Screen Shot 2014-04-15 at 12.06.32 AM

Screen Shot 2014-04-15 at 12.07.15 AM

[FUTURE] 6) We didn’t add it in yet for a number of reasons, but plan to complete the loop and notify back with a tweet saying “Here’s your blog link!”.

More infos:

[1] So why make the REST call to vCO instead of AppD directly? 

Good question!   Doing it this way can be considered best practice because it can give you greater flexibility in processing all the data and lets you monitor the process and better trap erros. This way you are not relying on the 3rd party system entirely but can work around it and integrate better.

[2] VMware Solution Exchange?

Yes!  It’s great!  You can browse all of these canned & demo solutions like Websphere, Oracle, JasperReports, SharePoint, Jenkins, and Liferay to see how one could (or should) use AppD to automate deployments.

[3] Want to see the workflows and blueprints?

Maybe when I get some time to clean it all up. It’s kind of…messy right now.


@jrrickard / Jeremy Rickard for the help with the python script running the twitter bot.

@MarcoCaronna  for the AppD/vCO zen master skills simplifying the REST API

BitNami for the original WordPress blueprint here

Tagged , , ,

VMware vExpert 2014


Well all the other kids are posting it, so I guess I should.   I was tapped for a vExpert designation this year (probably for some of these mad science ramblings on the blog….just wait to see whats coming…)  I’m not really sure what this means yet but we’ll see!

Looks like companies use this as a marketing ploy to get people talking about their products.   I guess it works because there’s huge buzz on twitter about it.   PluralSight gave us a year service to their training videos which is cool.  Thanks PluralSight!


Tagged ,

VMware vCO Port and Account Quick Reference

Maybe it’s just me… but I can never for the life of me remember the correct ports and accounts to use for each part of VMware vCenter Orchestrator (vCO).  If it happens to me – it’s probably happening to a ton of people out there.

I hope this helps!

This is current as of vCO 5.5

Port 8281/https  Appliance Page.
Port 8283/https VCO Configuration, Default: vmware/vmware (or user vmware and PW set in OVF/OVA deployment)
Port 8480/https Appliance Configuration, Default: root/vmware (or user root and PW set in OVF/OVA deployment)
Client URL: https://fqdn:8281/vco/client/client.jnlp
Tagged ,