What is oAuth 2.0 ?

Almost every one of us is using services of Google, Microsoft, Facebook, Apple etc., every day, either directly or indirectly. We see in most of the common websites have a feature to Sign up with Google, Facebook, Apple etc., Such feature is possible mainly because of having oAuth and OpenID Connect.

Before we jump in what oAuth is, it's important to understand what exactly means by authentication and authorization.

Authentication : Process of verifying who a user is. E.g : Logging into your email with username and password

Authorization : Process of verifying what access does the user has. E.g : Verify if the user has access to delete the files shared in Google Drive

Let's assume that you have a personal locker at the house and only you have the keys to open it. Everyone in the house (including you) has a separate key only to open the main door of the house. In this scenario, opening the main door of the house with the keys is like Authentication. Once you get into the house, only you're authorized to open the personal locker and no one else. It's called as authorization.

oAuth 2.0 is an Open Standard Authorization (shortened as oAuth) protocol used for authorization and OpenID connect is an identity layer on top of oAuth for authentication.

It's becoming quite common to see options like below when we sign-up or login to any of the sites which we use day-to-day. When there's such option it means that, behind the scenes, these applications are using oAuth 2.0 and OpenID Connect.

Let's take a common example use case. Assume that you have a study course in coursera and you want the assignment deadlines of the course to be automatically pushed to Google calendar. You're already logged into Coursera.

In this case we need to provide a grant to Coursera to access our Google Calendar. And coursera can use this grant to create, read or modify events in Calendar.

The oAuth authorization URL from Coursera to Google looks something like below,

https://authorization-server.com/auth?response_type=code &client_id=****

&redirect_uri=https://redirected-app.com/callback

&scope=calendarreadandwrite

&state=****

Explanation:

• response_type = code : Application is expected to receive the authorization code once the authorization is successful
• clientid : The client id is an identifier of the calling app (coursera in our case). We can get the client id when we register our app/website with Google to access Google services through oAuth
• redirecturi : One of the URL which the developer has registered while registering the app in Google. Normally authorization server would reject the request if the URL doesn't match.
• scope : It represents the accesses requested by the caller. E.g : read, write
• state : It is used to store request specific data. The authorization server should normally return back the state parameter as such in response.

Once we enter the credentials, we will asked for the consent like below,

Once you provide the consent, a temporary grant is given to Coursera for accessing the Google Calendar. The grant is provided my means of authorization code sent from Google to Coursera.

Sample Authorization Code : 4/3gEmN1LypvXC6VVEqGGV_V9zKQyuw8g923UhNF3XnPZvgssIrBz0HHs0yAW3-q-XxLrKQIy7GXwgt4XnE94

Once the caller has the authorization code (coursera in our case), it should call the oAuth server again with the authorization code to exchange it for access and refresh tokens.
Access Token : Used to access the resource server (Google calendar in our case). This is normally sent in http header. For security reasons, the lifetime of access token is very short.

Sample Access Token : 1//04iGQnny2BEr-CgYIRAAGAQSNwF-L9IrPyxZ8840SE0MnwZeOkqXW21NBgT4YTmadS1q8Smqg9SFYmiuBzEDXN9VoDZxU8q3M
Refresh Token : Used to get a new access token from oAuth server when the existing one is expired. Refresh token has a longer life time. This token cannot be used to access the resources in resource server directly (like access token).

Sample Refresh Token : ya29.a0AfH6SMB4i8c43VmjYhdQ3tTksrh3EUVa6GwmQJzzSs0aggvTul0rpVqXS8H-qBPUFEMyE31USSMdtB3KQI6Wek6quNFh_2jV7l1D5l4vlbDyEwxJ-B7GlXMDyTy8BJD92cq9EFWTJRg87w7T0AWU_dcoJC3gGZs

So whenever coursera needs to access the Google Calendar, it should send the access token in header along with the http request to Google Server.

The scenario explaining the expired access token and accessing the resource is like below,

Google also provides an easy platform to understand and play with oAuth 2.0. It helps us to understand how exactly an oAuth authorization works.

Google Playground URL : https://developers.google.com/oauthplayground/

What is Serverless ?

    In the recent times after the tremendous evolution of cloud, I came across the word serverless / serverless computing / serverless architecture frequently. I was curious to know how an architecture does be named serverless and a computing be done without servers!! I got the answer when I was exploring about Cloud computing & learnt about AWS in recent times.

    Then I realized that Serverless is just a buzzword. It is not a technology or architecture. It just means that we do not care about infrastructure and manging servers. It is done by someone else for us. As simple as that!

Img Src : https://www.freelancinggig.com/

    Serverless computing is a dynamic execution model where the execution of the code is taken care by a cloud provider like Amazon AWS, Google Cloud, Microsoft Azure. The allocation of necessary server and resources are taken care automatically by the provider. The code is typically split into multiple functions and run inside the stateless containers. It’s because of this concept of split , sometimes serverless is referred as “Faas – Function as a service”. The major advantage is that we pay only for what we use.

    The evolution of serverless has started from the use of traditional on-premise servers, hiring remote server machines and then at last the containers.

    The developer will need to upload the code to the cloud and the provider takes care of the rest. The most fascinating thing which I have learnt recently regarding serverless is an offering from Amazon AWS - Lambda. It does a lot of magic & supports most of the commonly used programming languages.

Major advantages:

·       Cost efficient:

Pay only for what we use and nothing needs to be paid when the code is not executed (We just need to pay for any underlying resources used. E.g: Storage). Also, we pay per invocation call of the function, which is event driven.

·       High availability:

The availability is high & guaranteed since the code is put into the cloud. We can

take advantage of multiple availability zones / regions which is turn provides disaster recovery and so on.

·       Scalability:

Multiple instances of the function are created automatically when there is a need. Also, it is scaled down after a specific duration if the function is idle for a specific time.

·       No management of servers

No more worries about installing servers, configuring it, maintaining it, managing it with patches and so on.

    On a whole, we just write the code and it will be deployed & available for the end user in a matter of seconds! Serverless computing – it is just awesome!

What is a DataLake ?

In today's technological world with evolution of digitization, the speed at which we create the data is tremendous. A study last year shows that at-least 90% of data which exists in the world is generated in last 2 years. It has also been said that Data is the world's most valuable asset.

There is always a need for something new when we see that something is evolving at rapid speed. Once such which evolved in recent times is Data Lake. It's nothing but a lake of data which receives data from various sources and stores in it's natural format. Though it looks similar to the existing concept of Data Warehouse, Data Lake has it's own purpose to serve.

With growing competition in every field, there is always a need to provide he best to the consumers. The major use of Data Lake is that it receives data from every possible source & we can generate business value out of it (like making predictive analytics by gaining more insight).

How does a Data Lake different from Data Warehouse ?

A Data Warehouse contains relational & structured data from various known systems. The structure of the data is already defined and the received data is transformed before it is stored by means of Extract, Transform & Load (ETL).

A Data Lake stores data as such as it receives, both structured and unstructured. The stored data is transformed only when it is needed for processing by means of Extract, Load & Transform (ELT).

Image Credits : https://softcrylic.com/blogs/power-bi-and-power-query-elt-workflows-vs-etl/

With more historical data by using Data Lake, it's possible to have better forecasts. It's also possible to combine data from a numerous sources.

The potential challenge with Data Lake could be too much of data. It needs to be processed right with a right approach & strategy. Also security of the data should be taken into consideration.Normally a typical Organization would need both Data Warehouse & Data Lake but the ones which has Data Lake outperform , shows a study. 

Creating a Simple Bot with Dialogflow (Formerly Api.ai)

Dialogflow (formerly api.ai) is kind of a framework from Google which is used to interact with your product by building engaging voice and text-based conversational interfaces powered by AI.

Here are the simple steps to build a weatherbot which will fetch the realtime weather from Yahoo. Dialogflow interacts with Yahoo Weather by means of an Python API deployed in Heroku (with GitHub).

1. Open https://dialogflow.com/ and click sign up.
2. Sign in with your Google Account.
3. Once logged in, click on create Agent 

    4. Enter the name of the bot and click save

    5. Goto Intents and click Create Intent

6. Enter the intent name as weather-intent and enter the text under “User says” as required. This will be text which would be considered as input or recognized from your speech. Enter "yahooWeatherForecast" under action name (This is the action name in our python code available in Github). Click “Save”

7. Goto https://github.com/rameshjothimani/fulfillment-webhook-weather-python and click deploy to heroku

8. If you don’t have an account in heroku, you will be asked to Signup

9. After Signup, you will be asked for Appname and choose region.

10. Enter a Appname and click “Deploy App”

11. Enter https://weatherbotdemo.herokuapp.com/webhook in the browser, you should get as “Method Not Allowed” if the app is successfully deployed. We will get this error since GET is not allowed.

12. Enter the webhook URL under fulfillment in Dialogflow and click Save.

13. Click on Intents in Dialogflow and select “Use webhook” under fulfillment. Click Save.

14.  Enter the text in try it now as "how is weather in location_name ?"

Introduction to Data Science and Machine learning

Data science :

• Data means facts and statistics collected together for reference or analysis.
• Science means Systematic study of the structure and behavior of the physical and natural world through observation and experiment.

Data + Science = Data Science is nothing but extracting the knowledge and insights from the collected data in either structured or unstructured form.

Now you might think that Data Science is similar to Data Mining , which is the term which we are very familiar with for quite long time. The answer is “NO”. Data Science by itself contains Data Mining but Data Mining is not equivalent to Data Science.

Img Src : https://goo.gl/nTTGS2

Machine Learning :

As the name suggests, Machine Learning (ML) is nothing but training a Program to perform better with the experience. It’s like the more you make the machine to practice , the more efficient it would be.

Tom Mitchell defined Machine Learning as “A computer program is said to learn from experience E with respect to some task T and some performance measure P, if its performance on T, as measured by P, improves with experience E.”

A simple example would be the recommendations in Online sites. Let’s say you are trying to buy a new iPad, then you are most likely to buy a cover for it. So the cover for your iPad will be shown in the recommendations.

But you should keep in mind that, it’s not always possible to train a machine with the set of expected inputs/results. There are different types of Machine Learning to make the machines learn or to train it ;)

Machine Learning is done with the help of Data Science. Machine Learning algorithms learn from Data, so basically we are implementing Data science in ML.

Does Machine Learning is another term for Artificial Intelligence ? My answer would be “NO”. Machine Learning is a part of Artificial Intelligence which works well with Data Science.

The different Machine Learning Categories are,

• Supervised Learning
• Unsupervised Learning
• Reinforcement Learning

Machine Learning & DataScience are the current buzz words in industry which has given birth to new kids like Chatbots, IOT (Google Home, Amazon Alexa...) etc.,etc.,

What is meant by Microservices ?

Microservice is an approach to develop an application by putting a number of smaller services together. The services are built based on the functionality / business need of the application and it simply means a small independent service. This makes one to understand the application much easier.

Micro-service could be considered as subset of SOA (Service Oriented Architecture). Micro-services are independently deployable whereas SOA is group of micro-services. In simple words, the components may belong to the same application in SOA but Micro-services are independent services.

The major advantage of microservices is greater scalability and easy maintainability. Each service has it's own data model,database etc., and can be deployed independently.When a service is down or not working, it impacts only a specific functionality and not the entire system.

A simple example of micro-service implementation is Shopping Cart. The service take care of only the task of placing the Order and it does not have to worry about tasks regarding customer details, payment and so on.

Infact there is no difference between Microservices architecture and Service Oriented Architecture (SOA). Microservices are not completely new, it's just a new term for existing implementation.

As we are moving towards cloud and more applications are being deployed in Cloud, Microservices play a key role.Normal server side applications would handle business logic, provide services to client and perform database operations. Any minor change in the application would need a complete redeployment of the whole application and it would bring the complete application down when it's being redeployed. Extensive resources are also needed to scale such applications.

Cloud applications should be failure resistant. And making normal server side applications to be failure resistant is tedious and requires extensive resources. Microservices makes this task easier. Incase of failure, it's easy to fix or redeploy the specific service. It will impact only one specific task of application and rest of the application will resist and runs undisturbed which means higher availability.

Microservices Architecture enable continuous delivery and deployment. Microservices are emerging rapidly and is becoming one the preferred styles for developers and architects.

What is Internet Of Things (IoT) ?

Internet of Things , the name by itself suggests that it is nothing but a group of objects connected by means of internet. The term is commonly referred to as IoT. It's internetworking of physical devices which includes household appliances, vehicles and so on. The devices collect and transmit the data over the internet as needed.

IoT makes one to sense the status and control the objects remotely. A simple example would be like, Let's say you want to switch off all the lights and fans when you exit from home, you just need to program in such a way. All the lights and fans will get switched off when you exit !

Image Credit : https://www.fiware.org

IoT makes our work more simpler. Another example is like, let's assume that the road which you usually travel has very high traffic. You might be able to get instruction in advance to get an alternative route to reach the destination much faster.

It provides tremendous opportunities which we can't even imagine since all the devices are connected by means of Internet. There are multiple companies working on IoT concept and Google is one major investor working on it. There are multiple other appliance companies which are working with Google in implementing IoTs.

Google is primary focusing on Home Automation, Driverless cars and so on. Most of the Google's IoT inventions are available only in United States and is expected to be available across the world soon.

Though IoT is a buzz word in recent times and named couple of years back, the IoT development was there since decades. IoT is also closely associated with machine learning, artificial intelligence, big data and many others. Internet of Things is one of the biggest transformations we have in recent times and for sure it will rule in future.

Amsterdam Marathon 2016 - My Experience

Running a marathon is one of the most exciting experience of my life. I believed that I couldn’t and wouldn’t be able to run a half-marathon ever in my life but I broke my belief. I am not an athlete or an avid runner till I have started thinking of running a half-marathon.

I have started my practice three months before and I was inspired by my friend Mr. Gopinathan who is an avid runner. During my initial days I was not even able to run 1 KM but I practiced everyday morning with short distances. On weekends I tried to run slightly longer distances than I usually do in weekdays. Gradually I started covering more distance as the practice continued and it has boosted my confidence.

I was able to cover 17 KMs in one and half months. At some stage I thought that I would not be able to run a half-marathon this year but still I kept practicing. A month before the event, I was able to cover 20 KMs. A bunch of our friends then registered for Amsterdam Marathon.

Running a marathon was a dream. I felt more confident with my current performance and have tested myself in Brussels Marathon on 2nd October 2016 which is just two weeks before Amsterdam Marathon. I couldn’t believe myself , I have completed it in 2hr 30mins.

Brussels Marathon - Finish

Then came the day for Amsterdam Marathon at historic Olympic stadium. The weather was hot than usual and we arrived at the venue by morning. The half-marathon event is scheduled to be at noon. We were all super excited. 

Then it’s the time to start half-marathon. A bunch of our friends geared up to the start line (Of course after a lot of snacks ;) ) equipping our self with nice Group T-Shirts, BIB & with a bunch of accessories. The race began and we started to carry a bunch of energy and excitement in exploring Amsterdam. The support from the crowd was outstanding.

After I ran for about 18 KMs , I was completely exhausted. The weather became more hot than usual. When I have just started to walk, there were a bunch of kids who just read my name in my BIB and shouted “Come on Ramesh…!” .I don’t know where the energy came from ! I started to run again, slowly and steadily.

Amsterdam Marathon - Finish

Finally I reached the Olympic stadium. Looking at countdown every 100 meters. The finish mark began to glitter and seemed to be waiting for me ! Yes, finally I reached and completed Amsterdam Half Marathon in 2 h 31min. The happiness, confidence, energy and the excitement I got after crossing the finish line is incredible.

Trust me ! Running marathons are super fun ! Definitely you will feel like a Champion !

A Day @ Switzerland !

Swiss is one of the most beautiful (and ofcourse expensive :) ) place on the earth with picturesque mountains, . I have been there twice and have visited Lucerne, Interlaken, Niesen & Titlis. I would recommend to visit Swiss in months of around April / May. It would be full of snow and you will have a wonderful time.

The currency of Swiss is Swiss Francs – CHF but Euro (EUR) will be accepted almost everywhere. One CHF is more or less is equivalent to an EUR.

You can also withdraw Euro from Swiss ATMs (I guess not all). I would recommend to use your EUR Credit / Debit cards for all your purchases.

Niesen :

Niesen, also called as Swiss Pyramid is a mountain in Switzerland. We have been there around 10 clock in morning. There is a short beautiful ride in funicular to reach the top of Niesen. From the top of Niesen we will get breathtaking views. If you visit during summer, you would have an option of Paragliding . It would be wonderful to watch the beauty of nature by flying high. If you visit during winter, then the top of Niesen would be full of snow and you would not have option of paragliding. There is also a restaurant at the top of Niesen ! We were back from Niesen around 2 o clock in afternoon in funicular. A nice place for photo freaks.

Lucerne :

We have been in Lucerne by evening. There are a lot of shops (mainly watches) , Lion Monument, Lucerne lake and Chappel bridge. You should definitely take a boat ride in Lucerne lake which would last for about one hour. One of the most beautiful lakes I have ever seen.

Lion monument is a sculpture of a wounded lion which looks very lovely.

Chapel bridge is also one of the must-see attraction to visit and it's best to visit by late evening. The bridge is photogenic as well. The photos looked more attractive from whichever position we click. There are many photo paintings and wooden boards in the bridge.

Image Credit : http://travelblog.viator.com/visiting-zurich-switzerlands-biggest-city/

One of the famous tamil song New York Nagaram from movie Sillunu Oru Kadhal is shot @ Lucerene.

Problem connecting to Amazon AWS EC2 Instance from Local

Most of the Amazon AWS EC2 beginners face the problem while connecting to Amazon EC2 instance from their local. The common error faced is “Disconnected : No supported authentication methods available (server sent :publickey) “

You would have already got the keypair from EC2 Management console and have used putty gen to generate private key file(PPK) but still an error is thrown. The reason is that the default user name is not correct.

The default user names are as follows,

• Amazon Linux: ec2-user
• Ubuntu: ubuntu
• Debian : admin

Don’t forget to add a security group rule to allow port 80 and 443 inbound. It's done and you ll be able to connect to your EC2 instance now.

I was fighting with this issue for long time and got the help from stack-overflow to get it resolved. The question can be referred @ Stack Overflow

How to customize Attribution widget in blogger and remove “Powered by blogger” in the footer of your blog?

If we are using a blogger template , the normal footer text which we could see is "Powered by blogger". Only the option for appending the text will be provided normally , but there is a way to remove the default text and make it to our customized one.

Follow the below steps ,

1. Login to blogger and select template menu on left

2. Click Edit Template below the template shown

3. Click inside the HTML Codes shown and press Ctrl+F

4. Search for the code below,

         <b:widget id='Attribution1' locked='true' title='' type='Attribution'>

Replace it with the following code,

                  <b:widget id='Attribution1' locked='false' title='' type='Attribution'>


The change is that locked attribute is changed to false from true.

5. Search for the below code as well,

                   <b:section class='foot' id='footer-3' showaddelement='no'>

Replace it with the following code,

                  <b:section class='foot' id='footer-3' showaddelement='yes'>

6. Save the template

7. Click the layout menu on the left

8. You can find the attibution element at the bottom of your page

9. Click on edit. Now you will get the option remove in the popup which will appear

10. Click remove button.

The default text will be removed at the footer and you can add any text as you wish !