Wednesday 16 April 2014

Learning REST services.

What is Rest Full Service?

REST stands for Representational State Transfer. (It is sometimes spelled "ReST".) It relies on a stateless, client-server, cacheable communications protocol -- and in virtually all cases, the HTTP protocol is used..

The idea is that, rather than using complex mechanisms such as CORBA, RPC or SOAP to connect between machines, simple HTTP is used to make calls between machines.

RESTful applications use HTTP requests to post data (create and/or update), read data (e.g., make queries), and delete data. Thus, REST uses HTTP for all four CRUD (Create/Read/Update/Delete) operations.


Much like Web Services, a REST service is:

Platform-independent (you don't care if the server is Unix, the client is a Mac, or anything else),
Language-independent (C# can talk to Java, etc.),
Standards-based (runs on top of HTTP), and
Can easily be used in the presence of firewalls.


How Simple is REST?

querying a phonebook application for the details of a given user. All we have is the user's ID.

Using Web Services and SOAP, the request would look something like this:

<?xml version="1.0"?>
<soap:Envelope
xmlns:soap="http://www.w3.org/2001/12/soap-envelope"
soap:encodingStyle="http://www.w3.org/2001/12/soap-encoding">
<soap:body pb="http://www.acme.com/phonebook">
<pb:GetUserDetails>
<pb:UserID>12345</pb:UserID>
</pb:GetUserDetails>
</soap:Body>
</soap:Envelope>


And with REST? The query will probably look like this:



This URL is sent to the server using a simpler GET request, and the HTTP reply is the raw result data -- not embedded inside anything, just the data you need in a way you can directly use.
A nice analogy for REST vs. SOAP is mailing a letter: with SOAP, you're using an envelope; with REST, it's a postcard. Postcards are easier to handle (by the receiver), waste less paper (i.e., consume less bandwidth), and have a short content. (Of course, REST requests aren't really limited in length, esp. if they use POST rather than GET.)


REST can easily handle more complex requests, including multiple parameters. In most cases, you'll just use HTTP GET parameters in the URL.

For example:


As a rule, GET requests should be for read-only queries; they should not change the state of the server and its data. For creation, updating, and deleting data, use POST requests. (POST can also be used for read-only queries, as noted above, when complex parameters are required.)

In a way, this web page (like most others) can be viewed as offering services via a REST API; you use a GET request to read data, and a POST request to post a comment -- where more and longer parameters are required.

REST Server Responses

A server response in REST is often an XML file; for example,

<parts-list>
<part id="3322">
<name>ACME Boomerang</name>
<desc>
Used by Coyote in <i>Zoom at the Top</i>, 1962
</desc>
<price currency="usd" quantity="1">17.32</price>
<uri>http://www.acme.com/parts/3322</uri>
</part>
<part id="783">
<name>ACME Dehydrated Boulders</name>
<desc>
Used by Coyote in <i>Scrambled Aches</i>, 1957
</desc>
<price currency="usd" quantity="pack">19.95</price>
<uri>http://www.acme.com/parts/783</uri>
</part>
</parts-list>


However, other formats can also be used; unlike SOAP services, REST is not bound to XML in any way. Possible formats include CSV (comma-separated values) and JSON (JavaScript Object Notation).

SOME REAL REST EXAMPLES

The Google Glass API, known as "Mirror API", is a pure REST API.
Twitter has a REST API
Flickr,
Amazon.com offer several REST services, e.g., for their S3 storage solution,
Atom is a RESTful alternative to RSS,
Tesla Model S uses an (undocumented) REST API between the car systems and its Android/iOS apps.


Key components of a REST architecture:



Resources, which are identified by logical URLs. Both state and functionality are represented using resources.
The logical URLs imply that the resources are universally addressable by other parts of the system.
Resources are the key element of a true RESTful design, as opposed to "methods" or "services" used in RPC and SOAP Web Services, respectively. You do not issue a "getProductName" and then a "getProductPrice" RPC calls in REST; rather, you view the product data as a resource -- and this resource should contain all the required information (or links to it).

A web of resources, meaning that a single resource should not be overwhelmingly large and contain too fine-grained details. Whenever relevant, a resource should contain links to additional information -- just as in web pages.

The system has a client-server, but of course one component's server can be another component's client.

There is no connection state; interaction is stateless (although the servers and resources can of course be stateful). Each new request should carry all the information required to complete it, and must not rely on previous interactions with the same client.

Resources should be cachable whenever possible (with an expiration date/time). The protocol must allow the server to explicitly specify which resources may be cached, and for how long.
Since HTTP is universally used as the REST protocol, the HTTP cache-control headers are used for this purpose.

Clients must respect the server's cache specification for each resource.
Proxy servers can be used as part of the architecture, to improve performance and scalability. Any standard HTTP proxy can be used.


Saturday 22 March 2014

Infrastructure as code. What!!! ? Why ???

I logged into my facebook account today and see a different look and feel. Honestly I was quite happy the way it was earlier , I hope facebook roles back. But as a Software Developer myself what amazes me is the ease with which facebook is able to roll such updates regularly and takes several decision based on sentimental analyses of how the new features are perceived.

I thought of digging in more and understand how do they do this.?

Just imagine a seasoned operations engineer who does regular tasks of provisioning and managing the plethora of environments.The tasks could be know all the admin consoles . The engineer now exactly when the next application release is due and also know when to expect the next update to the OS. The engineer is the master of his domain. Is that good enough in today's environment. With systems becoming virtualized and as developers have started practising Continuous Integration (CI), things have started to change. The number of environment and their instances you have to deal with has gone up by several orders of magnitudes. Developers now don’t release updates and new versions every few months; they are pumping out CI builds daily. In fact, multiple builds a day. All of them need to be tested and validated. That requires new environment instances to be spun up, fast. These builds also often come with configuration changes. Logging into consoles to make each one of these changes individually is no longer a viable option. Furthermore, the need for speed is critical.

So then how to deal with this situation ?

By modelling IT infrastructure and application delivery as code. When soft wares work reliably on code why not deployment and delivery.A code can Can Accelerate Time to Market, Code Can Encourage Innovation since the time being consumed in repetitive tasks can be taken care by the code.Code Can Scale.Code is Consistent.

Code provides a way to describe and automate infrastructure and processes. Infrastructure becomes testable, versioned and repeatable. It becomes part of Agile process.

By automating your application deployment and delivery vehicle, it becomes easy  to quickly roll out changes ..
FaceBook is great example of successfully implementing infrastructure as code .

Coming posts we will try to do this .


Wednesday 12 March 2014

What is a mobile app?

I got a query recently, “can you can implement devops for mobile apps?” . Which made me question, what is so special about mobile apps. Why do we have to think of devops for mobile aaps . So I thought of first try and find an answer to What is an mobile app?

What is a ‘Mobile ’ app after all?

“A mobile application is a software application designed to run on smartphones, tablet computers and other mobile devices. “

So then what are mobile devices.?

I thought of finding the definition for the same at Wikipedia.

“ A mobile device (also known as a handheld computer or simply handheld) is a small, handheld computing device, typically having a display screen with touch input and/or a miniature keyboard and weighing less than 2 pounds (0.91 kg).”

Really..? Well that's not what I thought it to be , even though the definitions can be treated as a subset of what I had in my mind.

I would further qualify that in today’s world one would limit the definition of mobile devices to ‘smart’ devices.Those with active, interactive software. It’s not just iOS vs. Android vs. Blackberry.And this is just the beginning. Televisions are now smart, though not untethered yet – they need to be connected to a cable or satellite network via a cable.All cars, trucks and two-wheelers have software embedded in their engines and electrical systems, not just the entertainment, navigation and comfort control systems. They can now tell you when the tires need more air, when the air filter needs to be changes, when you are drifting out of your lane and even parallel park themselves.

I thought of asking my teenage niece as to what is a mobile app. I got an answer ..one that is available at app store. You can download on you mobile.You can install easily. You can play games with. You don't need to be techy to use it. You keep getting updates regularly of the new improvements and more importantly everything is very simple with mobile apps. Really ? Yes that's it. Mobile apps have caught the imagination of everyone. What ever myths we had for desktop applications are no more there for mobile apps.


Mobile Apps are not Web Apps

So, are mobile apps just like web apps? No, they should be just given the same priority and attention as web apps, but they have special needs. Other than all the challenges related to supporting multiple devices with multiple OS versions and form factors, mobile apps are different from web apps in many ways. Focussing on differences related to deployment:

Mobile Apps need to go thru an App Store. This is an asynchronous step that takes a finite amount of time. Web Apps can be updates as needed or are at least within control of the enterprise.
Updating Mobile Apps is a ‘pull’. You cannot force a user to update their App when you want them to. So, unless you disable the ability for your backend to connect to older versions of the App, users can keep using them, even though updates are available.

No roll-back. Unlike web applications, mobile apps cannot be rolled back. You have to release a new version and ask users to upgrade to it.


So Considerations for writing a mobile application can be.
Understand the Potential of Mobile
Target Your Audience and Their Needs
Settle on an Objective
Measure Success
Test Regularly
Develop in Phases
Be Ready to Roll Out

So Now I ask my self a question . Do we need DevOps for mobile applications ? And the answer is most definitely YES!!.

In My next post I ll try to find answers for What does DevOps means for mobile Apps.


Monday 10 March 2014

IBM's Approach & realization of devOps.

The IBM solution for DevOps helps an organization collaborate across a broad group of stakeholders that includes not only development, but also line of business, clients, and operations teams

The value of DevOps can be illustrated as an innovation and delivery lifecycle, with a continuous feedback loop to learn and respond to customer needs.


The DevOps solution: What is needed?

Use a common set of practices based on lean thinking to maximize value through collaborative development and continuous testing (enabling continuous innovation), and eliminate any activity that does not directly benefit the customer.

Automate manual and overhead activities (enabling continu-ous delivery) such as change propagation and orchestration, traceability, measurement, progress reporting, etc.

Create a closed loop feedback mechanism (enabling continu-ous learning) in a customer facing environment

Use meaningful measurement and monitoring of progress (enabling continuous optimization) for improved visibility across the organization, including the software value delivery supply chain.

The following figure showcases the DevopOps capabilities.



The IBM solution to enable DevOps.



IBM provides an open, standards-based DevOps platform that supports a continuous innovation, feedback and improvement lifecycle, enabling a business to plan, track, manage, and auto-mate all aspects of continuously delivering business ideas. At the same time, the business is able to manage both existing and new workloads in enterprise-class systems and open the door to innovation with cloud and mobile solutions. This capability includes an iterative set of quality checks and verification phases that each product or piece of application code must pass before release to customers. The IBM solution provides a continuous feedback loop for all aspects of the delivery process (e.g., customer experience and sentiments, quality metrics, service level agreements, and environment data) and enables continu-ous testing of ideas and capabilities with end users in a customer facing environment.
IBM’s DevOps solution consists of the open standards based platform, DevOps Foundation services, with end to end DevOps lifecycle capabilities.


Friday 7 March 2014

DevOps: Delivery Model.

The ideas and principles guiding the DevOps concept are indeed noble. Bridging the gap between the development and operations arms of an organization is of immense benefit. It has been credited for the creation of dynamic, fortified, and functional systems, characterized by adaptability second to none, with a clearly stated agenda: the improvement of service delivery. Emphasizing collaboration and cooperation between the two often opposing groups is guaranteed to bolster quality of products, as each will be a result of an intricate blend of ingenious ideas. A marriage of operational and developmental concepts will, in many ways, ease the development of systems, leading to plausible SDLCs and the most effective of ALM policies.


Lets have a look at the continuous delivery work flow




A developer working on a Agile team. in a large project, having multiple teams developing different components of the software application you are building. Developer does his work, based on the user stories they and the team are working on. At the end of the day, He do a ‘private build’ of work to verify it builds and ‘deliver’ it to a team build server. Other team members also deliver their work. He ‘integrate’ work in the common build area and do an ‘Integration Build’.

Once the application is built, at the end of every Continuous Integration build, deliver it to the next stages in the application delivery lifecycle. Deliver it to the QA team for testing and then to the operations team (the Ops in DevOps) for delivery to the production system.
To test out that the builds coming out are ‘production-ready’ they should be delivered to a staging or test area which is production like..

Continuous Delivery doesn’t mean every change is deployed to production ASAP. It means every change is proven to be deployable at any time.

When continuously delivering software, one is not only validating the functionality and performance of the software being delivered and the environments it is being delivered to, but also the process of deploying the software itself. Deployment of the software is not as simple as copying some binaries over FTP. It involves file transfers, to multiple locations on potentially a complex set of nodes, but also involves configuration changes – to OS, databases and middleware. It also involves an orchestration of steps. One cannot simply do deployment steps in a mechanical linear manner. Middleware processes may need to be restarted after configuration changes. Services may need to be stopped before file transfers and then restarted, all in a coordinated orchestrated manner. Hence, continuous delivery allows for these processes to be tested and refined to ensure that when it comes to the final deployment to production, it is not the first time the team is executing the process(es). They are tested and proven to work.

In the flowchart above the Automation Agent for the delivery process takes care of deploying the resources to similar environments once the necessary artifacts are provisioned.

Coming next is to learn about DevOps Capabilities and IBM's tool chain to realizes devOps.

Wednesday 5 March 2014

DevOps Principles cont.... CTCM

Continuing to where we left in the last post are next two principles of DevOps.Continuous Testing /Continuous Monitoring which are facilitated by continuous integration .

Continuous Testing
Continuous Integration and Delivery are both (almost) meaningless without Continuous Test. Not having motoring and hence, not knowing how the application is performing in production makes the whole process of DevOps moot. What good is having an streamlined Continuous Delivery process is the only way you find out about your applications’ functionality or performance being below par is via a ticket opened by a disgruntled user?

Continuous testing reduces the cost of testing while helping development teams balance quality and speed. It eliminates testing bottlenecks through virtualized dependent services, and it simplifies the creation of virtualized test environments that can be easily deployed, shared, and updated as systems change. These capabilities reduce the cost of provisioning and maintaining test environments, and they shorten test cycle times by allowing integration testing earlier in life cycle.

Continuous Monitoring

In Production, the Ops team manages and ensures that the application is performing as desired and the environment is stable via Continuous Monitoring. While the Ops teams have their own tools to monitor their environments and systems, DevOps principles suggest that they also monitor the applications. They need to ensure that the applications are performing at optimal levels – down to levels lower than system monitoring tools would allow. This requires that Ops teams use tools that can monitor application performance and issues. It may also require that they work with Dev to build self-monitoring or analytics gathering capabilities right into the applications being built. This would allow for true end-to-end monitoring – continuously.

Putting The Four Continuous principles of devops
  • Continuous Integration
  • Continuous Delivery
  • Continuous Testing
  • Continuous Monitoring



Friday 28 February 2014

DevOps KeyPrinciples CICD.

In my earlier post I explained what is DevOps ? In this artcile I ll touch upon two important concepts of devops Continuous Integration /Continuous delivery.

Continuous Integration: – Agile in practice:


“Continuous Integration is a software development practice where members of a team integrate their work frequently; usually each person integrates at least daily - leading to multiple integrations per day. Each integration is verified by an automated build (including test) to detect integration errors as quickly as possible.”


  
The value that Continuous Integration brings to the table is that it forces developers and team of developers to integrate their individual work with each others as early as possible. This exposes integration issues and conflicts on a regular basis.

Principles of CI

  • Maintain a Single Source Repository
  • Automate the Build
  • Make Your Build Self-Testing
  • Everyone Commits To the Mainline Every Day
  • Every Commit Should Build the Mainline on an Integration Machine
  • Keep the Build Fast
  • Test in a Clone of the Production Environment
  • Make it Easy for Anyone to Get the Latest Executable
  • Everyone can see what's happening
  • Automate Deployment
Continuous Delivery – Extending Agile beyond Development:

Continuous Delivery takes Continuous Integration step further. After the application is built, at the end of every Continuous Integration build, deliver it to the next stages in the application delivery lifecycle. Deliver it to the QA team for testing and then to the operations team (the Ops in DevOps) for delivery to the production system. The goal of Continuous Delivery is to get the new features that the developers are creating, out to the customers and users as soon as possible. The goal being to Ensure applications are production-ready throughout the lifecycle and can be released at any time while minimizing rollback due to quality issues



Principles of CD

  • Validate on more production-like conditions earlier
  • Automate hand-offs/promotions to increase velocity through the different stages
  • Standardization on processes and assets between Dev and Ops
  • Automated monitoring and dashboarding of quality and performance against service level agreements at multiple stages