Once upon a time, Social Content Management and Social Business Systems were all the rage. And by once upon a time, I really mean 2010-2011. This pretty much equates to a bajillion years ago in Bay Area timelines. I suppose we needed something to bridge the gap between famed hype words of the time like Enterprise 2.0. So why bother with digging up the past? Well I don't think any of the major ECM vendors truly executed on Social Content Management.
This is partially due to the maturity of social media providers at the time and the way that consumers leveraged their platforms. Early on, social media was used to connect with like-minded people and to chronicle your daily adventures, thoughts, opinions, useless food pics, and the ubiquitous selfie...with or without the pouty duck face. Nowadays, social media is used to inspire entire movements and to inspire social change. What makes this possible, is the absence of confines and the lack of borders or controls. However, there are instances in which we need to create a snapshot in time and capture these social moments for compliance purposes or to drive business processes. To do this, we're going to integrate best-of-breed open platforms such as Activiti, Alfresco One, and Mule to create a supercharged Social Content Management solution.
Let's talk about the type of social content we want to capture. You remember all of those embarrassing moments when you drunkenly tweeted or posted something on Facebook? And for whatever reason, in your drunken haze the next morning, you thought it was best to expunge all of the evidence? How are you ever going to relive those infamous moments?! Well rest assured, because I'm happy to tell you that we can archive all of those precious moments in a DoD 5015.2 compliant system using Alfresco One and its Records Management module!!!
To capture all of this embarrassing social content, first we need to create a collaboration site in Alfresco One (or Alfresco Community) and prep it with some folder structure. This is where we're going to dump all of our social content from Twitter and manage it accordingly.
Now that we have a place to manage our social content, we'll leverage the Mule ESB and the Anypoint Studio graphical design environment to create a flow to capture it. Below is snapshot of our overall Mule flows which consist of an HTTP connector, Twitter connector, a For Each scope, a CMIS connector and a few transformers sprinkled in-between.
Let's begin by breaking the flow down piece by piece. The HTTP connector is responsible for initiating the flow by listening for HTTP requests and it will also provide an HTTP response after the flow has ended. Below is a screenshot of the HTTP Connector configuration. The important elements to take notice of are the host and port global properties.
After we define our HTTP connection properties, we'll want to provide a specific URL path so that we can call it from an HTTP client and so we can uniquely identify the endpoint from others that may be running on Mule.
Now lets tackle the Twitter step. Before we continue our configuration in Anypoint, we'll need to head over to https://apps.twitter.com/ to create a new app. Once created, switch over to the Keys and Access Tokens tab, and retrieve the consumer key, consumer secret, access token, and access token secret.
Now we'll take the consumer and access keys and secrets and drop them into our Twitter Connection Configuration. Note that you can put the values in the connection configuration directly OR you can place them in your mule-app.properties file and simply use a placeholder expression to refer to those properties.
In the Twitter step configuration, we'll need to choose the Search operation and use the [#json:hashtag] expression to populate the Query variable. This expression will pull a "hashtag" JSON property from the incoming HTTP request. One thing to note is that if you take a closer look at the available variables in the Search operation and compare them to the GET search/tweets Twitter documentation, the Count variable is completely missing! This means that your search result count will always be limited to the default of 15 tweets. So so so lame. Someone has already created an issue on the Twitter connector GitHub project here. But that was created about a year ago and hasn't received any love. If I'm feeling bored over the holidays, I'll update the Twitter connector to include the Count variable and will update the connector to use the latest and greatest twitter4j version.
Once we execute a search, we'll want to loop through all of the search results and perform a series of operations. To accomplish this in our Mule flow, we'll need to leverage a For Loop scope step. In that step, we need to specify a collection to iterate upon. This will be the payload from the Twitter search operation and the "tweets" JSON array element. Lastly, inside the For Loop, we're simply going to call our "create-twitter-content-flow" using a flow-ref step.
In the "create-twitter-content-flow," we'll need a Set Property transformer step which will set outbound properties to be used in subsequent steps in the flow. Specifically, we're going to leverage the twitter handle or screenName property, the unique id for the tweet, and the tweet text. Alternatively, you could use the Transform Message step which leverages the new DataWeave language and template engine.
The next step in the flow that we need to configure is a CMIS connector. In the CMIS Connection configuration, provide the ATOM binding CMIS URL, username, password, and select ATOM for the Endpoint property. Dropping back into the CMIS connector step, we're going to choose the "Get or create folder by path" operation and we're going to use "/Sites/yourSiteShortNameHere/My/Folder/Path/#[message.outboundProperties.screenName]" as the Folder Path value. This will dynamically create a new folder or retrieve an existing folder based on the predefined path and twitter handle.
The last step in our Mule flow will be another CMIS connector step that will create content based on the text from a tweet. To accomplish this, we need to choose the "Create document by path" operation, set the filename to be the tweetId, set the folder path of the folder we created in the previous step, set the Content Reference to the tweetContent property, set the Mime Type to text/plain, set the Object Type to cmis:document, and finally set the Versioning state to MAJOR.
For simplicity, I've decided to not build any additional logic to detect if an existing piece of content with the same exact name in located in the defined folder path. So keep in mind that if you run this twice in a row, it will most certainly fail with an exception stating that content with the same name already exists in that folder. But this is just a demo, so don't even worry about it!
Now that we've completed our Mule flow, let's switch over to our Activiti BPM Suite instance to build a simple process to orchestrate our Mule flow. Before we build the process, we need to define an endpoint to connect to our Mule flow. To do this, navigate to the Identity Management app, switch to the tenants page, and then go to the Endpoints tab. From there, just add a new endpoint with the host and port that was configured in the Mule flow's HTTP connector.
Our endpoint has been created, so now let's go back to our dashboard and jump into the Kickstart application. The process we want to create can be done in either the BPMN editor or the more simplified Step Editor. Let's keep it simple and create a Step process called "Social Media Archive." The first thing we're presented with is a simple Process start step and we can choose to either Start the process by a User without any input or we can choose to Start the process by a User filling in a form. We're going to choose the latter and since we do not have a form for this particular use case in our forms library, we'll want to create a brand new form. Give your form a simple name and click create. In this form, we'll only need one field to capture the Twitter search text, so drag and drop the Text control from the left into the canvas on the right-hand side. Next, provide a name by clicking on the pencil icon and finally click the form's save button.
The first step in our process design is complete, so let's move onto the most important step. Click the plus icon after the start step and choose the REST Call step. In the Details tab, provide a nice name that you're proud of and switch over to the Endpoint tab. Here, we'll choose the POST as our HTTP Method, the endpoint we created earlier will be used for the Base Endpoint, and add "twitter-archive" will be used as the REST url value. The REST URL suffix will correspond with the suffix we created in our Mule flow's HTTP connector step. Switching over to the Request tab, we only need to add one JSON property to send along with our payload. We'll pull the hashtag value in from the form field created in the start form and we also want to make sure we use the "hashtag" JSON property name that corresponds with the Mule flow. Last, but not least, in the Response tab lets map JSON values for "query", "count" and "completedIn" to process variables.
The final step in our Social Media Archive process will be a simple human step to display the response from Twitter and the Mule flow. Click the plus icon after the REST Call step, and provide a simple name in the Details tab. Switch to the Form tab create a new form. Provide a name to your new form and drag and drop three Display Value form controls into the canvas. Click the pencil icon next to each of the Display Value controls. Instead of displaying a form field, switch to the Variable tab and choose the JSON process variables created in the REST Call step. Finally, validate the process and save it.
The last piece of configure for our solutions is to create a process application to encapsulate our Social Media Archive process. After navigating to the App page within the Kickstart app, click the Create App button and provide a new for your new process app. In the app definition page, you can configure a specific icon and color for the app. The most important piece of the app definition is to include your process. After you've included the Social Media Process click the save button, choose to publish the application, and finally save and close the editor. Finally, back on our dashboard, click the plus icon and add your newly created process application.
Now we can navigate to our Social Content Management app and to start our newly created process. Provide a hashtag and click the Start Process button. If everything worked correctly, after a few seconds, you should be able to refresh the browser and find a review step assigned to yourself. Navigate to the review step and check out the results.
There's only one thing left to do, so head back to Alfresco One's Share UI and check out the dynamically created folder structure based on twitter handles and the tweets stored inside each folder.
To recap, we've created a Social Content Management solution by leveraging Mule to pull content, transform data, and push it into Alfresco One. We also create a process in Activiti to orchestrate our Mule flow. By themselves, Activiti, Alfresco One, and Mule can be extremely powerful. But integrating all three together is on a whole other level.
Also check out the following YouTube video to see all of this working in real-time.
- Keep Calm and Nerd On!!!