Front-end Development in dotnet.sg

yeoman-bower-npm-gulp

The web development team in my office at Changi Airport is a rather small team. We have one designer, one UI/UX expert, and one front-end developer. Sometimes, when there are many projects happening at the same time, I will also work on the front-end tasks with the front-end developer.

In the dotnet.sg project, I have chance to work on front-end part too. Well, currently I am the only one who actively contribute to the dotnet.sg website anyway. =)

Screen Shot 2017-01-29 at 12.49.23 AM.png
Official website for Singapore .NET Developers Community: http://dotnet.sg

Tools

Unlike the projects I have in work, dotnet.sg project allows me to choose to work with tools that I’d like to explore and tools that helps me work more efficiently. Currently, for the front-end of dotnet.sg, I am using the following tools, i.e.

  • npm;
  • Yeoman;
  • Bower;
  • Gulp.

Getting Started

I am building the dotnet.sg website, which is an ASP .NET Core web app, on Mac with Visual Studio Code. Hence, before I work on the project, I have to download NodeJs to get npm. The npm is a package manager that helps to install tools like Yeoman, Bower, and Gulp.

After these tools are installed, I proceed to get a started template for my ASP .NET Core web app using Yeoman. Bower will then follow up immediately to install the required dependencies in the web project.

screen-shot-2017-01-28-at-9-03-10-pm
Starting a new ASP .NET Core project with Yeoman and Bower.

From Bower with bower.json…

Working on the dotnet.sg project helps me to explore more. Bower is one of the new things that I learnt in this project.

To develop a website, I normally make use of several common JS and CSS libraries, such as jQuery, jQuery UI, Bootstrap, Font Awesome, and so on. With so many libraries to manage, things could be quite messed up. This is where Bower comes to help.

Bower helps me to manage the 3rd party resources, such as Javascript libraries and frameworks, without the need to locate the script files for each resources myself.

For example, we can do a search of a library we want to use using Bower.

Screen Shot 2017-01-28 at 9.44.47 PM.png
Search the Font Awesome library in Bower.

To install the library, for example Font Awesome in this case, then with just one command, we can easily do it.

$ bower install fontawesome

The libraries will be installed in the directory as specified in the Bower Configuration file, .bowerrc. By default, the libraries will be located at the lib folder in wwwroot.

screen-shot-2017-01-28-at-10-08-44-pm
Downloaded libraries will be kept in wwwroot/lib as specified in .bowerrc.

Finally, to check the available versions of a library, simply use the following command to find out more about the library.

$ bower info fontawesome

I like Bower because checking bower.json into the source control ensures that every developer in the team has exactly the same code. On top of that, Bower also allows us to lock the libraries to a specific version. This will thus prevent some developers to download some different version of the same library from different sources themselves.

…to npm with package.json

So, now some of you may wonder, why are we using Bower when we have npm?

Currently, there are also developers supporting the act to stop using Bower and switch to npm. Libraries such as jQuery, jQuery UI, and Font Awesome, can be found on npm too. So, why do I still talk about Bower so much?

Screen Shot 2017-01-28 at 11.30.58 PM.png
Searching for packages in npm.

For ASP .NET Core project, I face a problem on referring to node_module from the View. Similar as Bower, npm will position the downloaded packages in a local folder also. The folder turns out to be node_module, which is on the same level as wwwroot folder in the project directory.

As ASP .NET Core serves the CSS, JS, and other static files from the wwwroot folder which doesn’t have node_module in it, the libraries downloaded from npm cannot be loaded. One way will be using Gulp Task but that one is too troublesome for my projects so I choose not to go that way.

Please share with me how to do it with npm in an easier way than with Bower, if you know any. Thanks!

Goodbye, Gulp

I first learnt Gulp was when Riza introduced it one year ago in .NET Developers Community Singapore meetup. He was then talking about the tooling in ASP .NET Core 1.0 projects.

Riza Talking about Gulp.png
Riza is sharing knowledge about Gulp during dotnet.sg meetup in Feb 2016.

However, about four months after the meetup, I came to a video on Channel9 announcing that the team removed Gulp from the default ASP .NET template. I’m okay with this change because using BundleMinifier to do bundling and minifying of CSS and JS now without using Gulp because using bundleconfig.json in BundleMinifier seems to be straightforward.

Screen Shot 2017-01-28 at 11.59.18 PM.png
Discussion on Channel 9 about the removal of Gulp in Jun 2016.

However, the SCSS compilation is something I don’t know how to do it without using Gulp (Please tell me if you know a better way. Thanks!).

To add back Gulp to my ASP .NET Core project, I do the following four steps.

  1. Create a package.json with only the two compulsory properties, i.e. name and version (Do this step only when package.json does not exist in the project directory);
  2. $ npm install --save-dev gulp
  3. $ npm install --save-dev gulp-sass
  4. Setup the generated gulp.js file as shown below.
var gulp = require('gulp');
var sass = require('gulp-sass');

gulp.task('compile-scss', function(){
    gulp.src('wwwroot/sass/**/*.scss')
        .pipe(sass().on('error', sass.logError))
        .pipe(gulp.dest('wwwroot/css/'));
})

//Watch task
gulp.task('default', function() {
    gulp.watch('wwwroot/sass/**/*.scss', ['compile-scss']);
})

After that, I just need to execute the following command to run gulp and changes made to the .scss files in the sass directory will trigger the Gulp Task to compile the SCSS to corresponding CSS.

$ gulp

There is also a very detailed online tutorial written by Ryan Christiani, the Head Instructor and Development Lead at HackerYou, explaining each step above.

Oh ya, in case you are wondering what is the difference between –save and –save-dev in the npm commands above, I like how it is summarized on Stack Overflow by Tuong Le, as shown below.

  • –save-dev is used to save the package for development purpose. Example: unit tests, minification.
  • –save is used to save the package required for the application to run.

Conclusion

I once heard people saying that web developers were the cheap labour in software development industry because they are still having the mindset that web developers just plug-and-play modules on WordPress.

After working on the dotnet.sg project and helping out in front-end development at work, I realize that web development is not an easy plug-and-play job at all.

Picking Up SCSS

sass-web-compiler-visual-studio-pluralsight

Last week, during our work discussion, we came to this point where we argued if “fast first, slow later” or “slow first, fast later” is suitable in our working environment.

In startup mode, everything comes at you quickly, and you tend to react fast. So in the first two years of setting up the Innovation Team in Changi Airport, our software development team had been working very hard and very fast to meet the deadline. Now, our company is switching from startup mode to scale-up mode where we need to shift towards doing things right more often than doing things fast.

Hence, we are working on setting up a set of suitable development and design principles in our development team. Applying SCSS to refactor our CSS is part of this time-consuming, difficult, and tiring process.

Installing Web Debugger in VS2015

After the introduce of Web Essentials 2015, features such as compiling SCSS files have been moved to another extension called Web Compiler in Visual Studio.

Hence, to get started in VS2015, we need to first download Web Compiler via Tools > Extensions and Updates.

Installing Extensions in VS2015.png
Installed Web Compiler in Visual Studio 2015

We will need to restart Visual Studio after the installation. Once the Visual Studio is restarted, we then can start using SCSS in our web projects.

By using Web Compiler, every time we save the .scss file, it will auto compile it to be a corresponding .css file (with minified version as well!).

Another feature that I like in this extension is that Visual Studio will specify whether the SCSS files are “Compiled successfully” or there is any SCSS error, as shown in the screenshot below.

SCSS Error Reporting in VS2015.png
Visual Studio will provide friendly error messages for SCSS too!

Refactor CSS into SCSS

Previously, besides using CSS from Bootstrap, we mostly handcrafted our CSS. Recently, it had become quite hard to maintain. So I started to refactor the CSS files from one of our web projects into SCSS.

Firstly, I created a new set of blank SCSS files while keeping the existing CSS files untouched. Secondly, I change the CSS reference of the website to use the new CSS files generated by the Sass pre-compiler. By doing this, I can choose to slowly refactor the existing CSS.

Change I Love #1: Introduction of Variables

Taking just brand colour as an example, currently our CSS files have it all over the place. The same shade of blue appears a lot of times. It is incredibly hard and time consuming to make changes in our web projects using plain CSS.

Previously, for example, we have the following CSS.

.btn-main {
    background-color: #28c8f0;
    border-color: #28c8f0;
}

The primary colour #28c8f0 is used in other classes throughout the whole CSS. Hence, we can just define it as a variable $primary-color: #28c8f0; and then use it

.btn-main {
    background-color: $primary-color;
    border-color: $primary-color;
}

In the future, if we want to change the primary colour to another colour, we just need to change it at one place without worrying if we miss out any part of the CSS not updated.

Change I Love #2: DRY with Mixin

Don’t Repeat Yourself (DRY), if we are using plain CSS, we normally find ourselves reusing the same set of codes throughout the CSS files. So, by using mixins in SCSS, there will always be one and only one set we need to remember and reuse.

Before using SCSS:

.customized-width-250 {
    margin-top: 4px;
    border: 1px solid #ffffff;
    border-radius: 5px 5px 5px 5px;
    font-weight: bold;
    height: 30px;
    min-width: 250px; 
}

.customized-width-120 {
    margin-top: 4px;
    border: 1px solid #ffffff;
    border-radius: 5px 5px 5px 5px;
    font-weight: bold;
    height: 30px;
    min-width: 120px; 
}

.customized-width-60 {
    margin-top: 4px;
    border: 1px solid #ffffff;
    border-radius: 5px 5px 5px 5px;
    font-weight: bold;
    height: 30px;
    min-width: 60px; 
}

Now, by using mixin, we can easily remove the duplicates for easy maintenance.

@mixin customized-controls ($width) {
    margin-top: 4px; 
    border: 1px solid #ffffff; 
    border-radius: 5px 5px 5px 5px; 
    font-weight: bold; 
    height: 30px;
    min-width: $width;
}

.customized-width-250 {
    @include customized-controls(250px);
}

.customized-width-120 {
    @include customized-controls(120px);
}

.customized-width-60 {
    @include customized-controls(60px);
}

Change I Love #3: Loops and Conditional

On our website, we need to display representative image for each of the countries available on the portal.

If we are using plain CSS, we need to do the following for each country. For example, for Australia, we have the follows.

.country-box-australia {
    background-image: url("/images/device-country-australia.png");
}

Now we have 9 countries on our portal. So we need to repeat the lines above for 9 times. If the images are moved to another folder, then we need to update the CSS at 9 places.

In SCSS, we can use list and each loop to make the CSS more readable.

$portal-countries: australia, france, hong-kong, japan, malaysia, new-zealand, south-korea, taiwan, thailand;

@each $country in $portal-countries {
    .country-box-#{$country} {
        background-image: url('/images/device-country-#{$country}.png')
    }
}

As you see above, it also makes use of Interpolation #{} to make the code even cleaner.

Change I Love #4: Color Functions

This is helpful especially when we do the hover effect for buttons. Previously, we always needed to ask the Design Team to give us two colour codes for buttons. One for non-hover and one for hover.

So with the Color Functions in SCSS, we can now do as follows.

a {
    text-decoration: none;
    color: $primary-color;

    &:hover, &:focus {
        text-decoration: none;
        color: darken($primary-color, 20%);
    }
}

We then can have a consistent look-and-feel throughout the whole website.

Oh ya, the & character above is used to reference parent selector.

Change I Love #5: Partials

We can also have partials by starting the name of the partials with an underscore.

Because of partials, we can organize our SCSS files properly according to their functionality.

Conclusion

I believe that now given the fact that our company is already in a scale-up mode, if we keep doing everything in a hacking way, we will easily end up with technical debt soon. Having technical debt means that we will need to spend extra development work in the future because the best overall solution is not implemented in the beginning.

That is why I always welcome opportunities to learn and improve my skills. This includes learning from my teammates via our countless conversations because the conversations kept me inspired and kept me going. The team had made me a better developer. Picking up SCSS is one of the examples and it is only the beginning.

Learning Materials

Exploring Azure Search

azure-search-meetup.png

Last year, Riza shared about his small little algorithm to do smart auto complete in WPF in Singapore .NET Developers Community March meetup. Riza has his project for this, SmartSuggestions, available on Github. What it does is that it will prompt user for smart suggestion of typos and find the similar words for suggestion.

meetup-riza-smart-suggestion
Riza Marhaban is sharing his SmartSuggestion algorithm to the audience during the community meetup. (Photo Credit: Singapore .NET Developers Community)

I find his program to be very interesting. In fact, I did a similar task when I was working in Easibook as well. By calculating the Levenshtein Distance of user input and the records in database, the small JavaScript code I wrote is able to suggest the places even user keys in the place name wrongly.

Soon after Riza’s talk about his SmartSuggestion, I read the announcement of general availability of Azure Search from Microsoft team.

azure-search-generally-available
Azure Search is generally available!

Azure Search

Azure Search is a fully managed search-as-a-service in Microsoft Azure. It offers scalable full-text search for the program. Hence, with its help, developers do not need to re-invent the text-searching capability in their programs and websites.

Azure Search currently provides two ways of querying text. One is using Simple Query Syntax where user can do keywords searching, phrase searching, suffix searching, etc. AND/OR/NOT operator is also available for use.

Another way of querying will be Lucene Query Parser. What interests me the most in Lucene Query Syntax is the use of Damerau–Levenshtein Distance in its Fuzzy Search, which does more than the Levenshtein Distance that only allows insertion, deletion, and substitution operations.

Try It Out!

In order to try out this feature, I have decided to create a demo program to test its functionality.

In this program, I use the event data from the .NET Developers Community Singapore to demonstrate how Azure Search works. To do this, I have to integrate with the Meetup APIs in this program.

Currently, this demo application covers the following features in Azure Search.

  • Create Azure Search index;
  • Data upload;
  • Keywords query in both Simple Query Syntax and Lucene Query Syntax.

Here are some of the screenshots of querying using Azure Search.

For example, if I’d like to find out what the talks covering topic about Visual Studio are, I can just simply search by “visual studio” as a phrase, as shown in the following screenshot.

azure-search-phrase-search
Phrase Searching in Azure Search

Or let’s say a user wants to search the meetup events about “Xamarin” but he doesn’t know its correct spelling is either Xamarin or Zamarin. So he can do a Fuzzy Search by keying in “Zamarin~”. Take note of the tilde “~” symbol at the end of the word. It means the search of the word will be done using Fuzzy Search.

azure-search-fuzzy-search
Fuzzy Search in Azure Search

Holiday and Coding

Christmas is a public holiday in Singapore. Since Christmas is on Sunday, I get a day off on Monday. So besides taking a rest in my room, I did a quick research on Azure Search. It’s kind of fun because it helps me to learn new things which I don’t have chance to explore during work.

search-with-ease.png
With Azure Search, we can now search with our minds at east. (Image Credit: Re:Zero Kara Hajimeru Isekai Seikatsu, KissAnime)

Anyway, I have uploaded my demo program project to Github. Feel free to check it out!

github-azure-search-demo

Burger and Cheese

xamarin-cognitive-services-android-voicetext

As a web developer, I don’t have many chances to play with mobile app projects. So rather than limit myself to just one field, I love to explore other technologies, especially mobile app development.

Burger Project: My First Xamarin App

Last month, I attended a Xamarin talk at Microsoft Singapore office with my colleague. The talk was about authentication and authorization with social networks such as Facebook and Twitter via Azure App Service: Mobile App.

Ben Ishiyama-Levy is talking about how Xamarin and Microsoft Azure works together.
Ben Ishiyama-Levy is talking about how Xamarin and Microsoft Azure works together.

The speaker is Ben Ishiyama-Levy, a Xamarin evangelist. His talk inspired me to further explore how I could retrieve user info from social network after authenticating the users.

Because I am geek-first and I really want to find out more, so I continue to read more about this topic. With the help from my colleague, I developed a simple Xamarin.Android app to demonstrate the Authentication and logged-in user’s info retrieval.

The demo app is called Burger and it can be found on my Github repository: https://github.com/goh-chunlin/Burger.

Challenges in Burger Project

Retrieving user's info from social network.
Retrieving user’s info from social network.

In Burger project, the first big challenge is to understand how Azure App Service: Mobile App works in Xamarin. Luckily, with the material and tutorial given in the Xamarin talk from Ben, I was able to get a quick start on this.

My colleague also shared another tutorial which is about getting authenticated user’s personal details on Universal Windows Platform (UWP). It helps me a lot to understand about how mobile app and Azure App Service can work together.

My second challenge in this project is to understand Facebook Graph API. I still remember that I spent quite some time finding out why I could not retrieve the friend list of a logged-in Facebook user. With the introduction of the Facebook Graph API 2.0, access to a user’s friends list via /me/friends is limited to just friends using the same app. Hence after reading a few other online tutorials, I finally somehow able to get another subset of a user’s friends via /me/taggable_friends.

In this project, it’s also the first time I apply Reflection in my personal project. It helps me easily get the according social network login class with a neat and organized code.

microsoftdeveloperday
Microsoft Developer Day at NUS, Singapore in May 2016

Cheese Project: When Google Speech Meets MS LUIS on Android

Few months ago, I’m fortunate to represent my company to attend Microsoft Developer Day 2016 in National University of Singapore (NUS).

The day is the first time Microsoft CEO Satya Nadella comes to Singapore. It’s also my first time learn about the powerful Cognitive Services and LUIS (Language Understanding Intelligence Service) in Microsoft Azure in Riza’s talk.

presentation
Riza’s presentation about Microsoft Cognitive APIs during Microsoft Developer Day.

Challenges in Cheese Project

Everyday, it takes about one hour for me to reach home from office. Hence, I will only have two to three hours every night to work on personal projects and learning. During weekends, when people are having fun out there, I will spend time on researching about some exciting new technologies.

There are many advance topics in LUIS. I still remember that when I was learning how LUIS works, my friend was actually playing the Rise of the Tomb Raider beside me. So while he was there phew-phew-phew, I was doing data training on LUIS web interface.

luis
Microsoft LUIS (Language Understanding Intelligence Service) and Intents

Currently, I only worked on some simple intents, such as returning me current date and time as well as understanding which language I want to translate to.

My first idea in Cheese project is to build an Android app such that if I say “Please translate blah-blah to xxx language”, the app will understand and do the translation accordingly. This can be quite easily done with the help of both LUIS and Google Translate.

After showing this app to my colleagues, we realized one problem in the app. It’s too troublesome for users to keep saying “Please translate blah-blah to xxx language” every time they need to translate something. Hence, recently I have changed it to use GUI to provide language selection. This, however, reduces the role played by LUIS in this project.

voicetext
VoiceText provides a range of speakers and voices with emotions!

To make the project even more fun, I implemented the VoiceText Web API from Japanese in the Android app. The cool thing about this TTS (Text-To-Speech) API is that it allows developers to specify the mood and characteristic of the voice. The challenge, of course, is to read the API written in Japanese. =P

Oh ya, this is the link to my Cheese repository on Github: https://github.com/goh-chunlin/Cheese. I will continue to work on this project while exploring more about LUIS. Stay tuned.

languagelist    googlespeech    SuccessfullyTranslated.png

After-Work Personal Projects

There are still more things in mobile app development for me to learn. Even though most of the time I feel exhausted after long work day, working on new and exciting technologies helps me getting energized again in the evening.

I’m not as hardworking as my friends who are willing to sacrifice their sleep for their hobby projects and learning, hence the progress of my personal project development is kind of slow. Oh well, at least now I have my little app to help me talking to people when I travel to Hong Kong and Japan next year!

Growth Hacker: Hybrid of Marketer and Programmer

Growth Hacker Marketing and Growth Hacking Handbook

I don’t know why people like to call software developers as hackers. Even though calling them hackers is way better than using other names like ninjas or code monkeys, I think it’s still not appropriate to call developers hackers. So, I find it quite weird when I met someone who is Growth Hacker. Err… Hack what?

Growth Hacker

Andrew Chen, an investor for tech startups, describes growth hacker as

a hybrid of marketer and coder, one who looks at the traditional question of “How do I get customers for my product?” and answers with A/B tests, landing pages, viral factor, email deliverability, and Open Graph.

A growth hacker is someone who has thrown out the playbook of traditional marketing and replaced it with only what is testable, trackable, and scalable. Their tools are e-mails, pay-per-click ads, blogs, and platform APIs instead of commercials, publicity, and money.
(Holiday, Ryan. 2013. Growth Hacker Marketing. New York : Penguin Group, 2013)

Hence, growth hacker plays an important role in startup or SME which has little to no resources. Growth hacker needs to make use of their programming skills to provide more scientifically way of understanding who customers are and where they are.

Developer + Marketer = Growth Hacker
Developer + Marketer = Growth Hacker

Growth hacking highlights the importance of Product Market Fit. Instead of expecting marketers to show a product that nobody wants, company with growth hacking mindset will now spend time on trying different way of improving product based on customer feedback.

Growth hacker tests the ideas by letting customers to try our different version of the product and then ask the customers what they like about the product. One way to receive feedback is by looking at conversion rate or by the number of Facebook likes received. Hence, this helps the company to publish a product which is worth marketing and has majority of its customers love to use.

Different Stages in Growth Hacking

I’m glad to have talked to people who are experienced in growth hacking. They recommended me two books to read. One is the Growth Hacker Marketing by Ryan Holiday.

In Ryan Holiday’s book, growth hacking basically has 3 stages as follows.

  1. Finding your growth hack;
  2. Going viral;
  3. Closing the loop: Retention and optimization.
3 stages in Growth Hacker Marketing.
3 stages in Growth Hacker Marketing.

Growth Hacking Tactics

Another book which is recommended to me to read is Growth Hacking Handbook written by Jon Yongfook. The book suggested 100 growth tactics which many startups have successfully applied over the last 2 decades.

I am not going to list all the 100 items here. So, I will just highlight some of them which I find to be interesting.

Tactic 01: The We Can’t Go Back Jack Hack

On the signup page or checkout page, disable or remove all navigational elements that would enable a user to go back to the previous page. This includes disabling your site logo, which is often linked to the homepage.

The reason of having this tactic is because preventing users from leaving a process is sometimes good enough to force them to complete the entire signup/booking process.

In the Amazon checkout page, user can't go back to the homepage by clicking on the logo.
In the Amazon checkout page, user can’t go back to the homepage by clicking on the logo. Instead they have to click on the tiny link at the bottom.

I don’t really like this idea because it’s sort of locking customers in your shop and then telling the customers that they can’t leave your shop until they make the payment. It creates a bad user experience.

Growth Hacking: Shut the door, release the dog and then lock the customers in our shop!
Growth Hacking: Shut the door, release the dog and then lock the customers in our shop!

Tactic 02: The Pre-filled Form Hack

Instead of forcing customers to buy our goods before leaving our shop, why don’t we just create a fast signup/checkout process?

One way of doing that is to make the form short and pre-fill form fields with information if we already have it, such as customer’s email address.

For example the checkout form below will scare customers away not because it’s too long but it also requests too much personal information from the customers without giving any explanation why the information is needed.

Booking ferry tickets from Singapore to Batam on Easybook requires you to enter every single passenger's name, gender, DOB, nationality, passport, and passport expiry date. On top of that, the system also requires you to manually specify the number of adults and children even though DOB of each passenger is already given.
Booking ferry tickets from Singapore to Batam on Easybook requires you to enter every single passenger’s name, gender, DOB, nationality, passport, and passport expiry date. On top of that, the system also requires you to manually specify the number of adults and children even though DOB of each passenger is already given.

Tactic 03: The As Seen On TV Hack

Users feel more comfortable when they know the products they are using now are something that other famous people have already used it. So having logos of media outlets who have mentioned about our product on our homepage is another form of social proof.

Drive.SG is using the As-Seen-On hack.
Drive.SG is using the As-Seen-On hack.

Tactic 04: The Multi Post Hack

This tactic is also quite straightforward. It basically means giving the option to post the website content to other social networks with just one click. That will help to push our product to a wider network via social networks.

Tactic 05: The Timebomb Hack

This is a pressure tactic where a time limit is set when a user is making critical decision such as a purchase. I don’t like to use this tactic in my ecommerce website because as a consumer, I prefer to choose what to buy without any pressure. I normally just quit the website when it forces me to finish a task in a very little time.

Customers needs to complete their transaction within 5 minutes on CurrencyBooking, a money changing platform.
Customers needs to complete their transaction within 5 minutes on CurrencyBooking, a money changing platform.

Tactic 06: The Winback Hack and the Negative Follow Up Hack

These two tactics are quite similar. The goal is to get feedback from users who are inactive and having incomplete transactions and incentivize them to return to the website.

A simple winback campaign can be automatically sending highly personalized email to those users who have not logged in for past 30 days. Normally, a winback campaign is our last chance at communicating with our users before they unsubscribe from us.

Sending email to customers who have signed up but do not buy any of our product is also important. That will help us to better understand why some customers do not want to buy from us (and sometimes it could be because of having bugs in the payment module).

The following is an email I received after subscribing an online service.

Hi Chun Lin,

I noticed you recently added our product to your cart but did not submit the order – wondering if you have any questions about our platform or pricing?

We’re always here to help so please do not hesitate to contact me! You can find my contact info, including my direct number, in the email signature below.

Direct number and contact info! Wow!

I don’t sign up with them in the end because I don’t really need their product at that moment. However, I’m still very impressed by their friendly email.

Tactic 07: The Intro Video Hack

If a picture is worth a thousand words, then a video is worth a million. Most people will find video content more interesting than standard text content. Hence, video content is useful to attract a significant number of inbound links and social shares for our website.

Recently, Google Search results page also displays Rich Snippet which contains information about the video embedded on the page. Hence, the Rich Snippet helps our web page to stand out from the other search results on the page. Therefore, users will be more inclined to click on the link pointing to our website.

Tactic 08: The Register to Save Hack

Instead of getting users to sign up first, we can choose to only ask users for their email after they have gone through some steps. This is to encourage users to try our product first to understand about the product better so that the eventual conversion to signup is easy.

Singapore Real Estate Exchange allows users to freely search, then requires login/signup when users want to shortlist the property.
Singapore Real Estate Exchange allows users to freely search, then requires login/signup when users want to shortlist the property.

Tactic 09: The Widget Hack and The Affiliate Program Hack

Instead of asking users to share our website via a link in text, why not giving them an embeddable widget to our product which can be easily added to any other website or blog? Then from there we can get those who embed our widget on their websites/blogs to participate in our affiliate program so that people are motivated to refer customers to our website.

Bus ticket search widget from BusOnlineTicket.com Affiliate Program.
Bus ticket search widget from BusOnlineTicket.com Affiliate Program.

Tactic 10: The Remarketing Tag Hack

Nowadays, on Facebook, there is an option to create Custom Audiences from our website which enable us to target our Facebook Adverts to only those who have visited our website. Alternatively, it also allows excluding existing customers so that we can focus on new customer acquisition campaigns.

IBM Connect 2015: SoftLayer and Bluemix

IBM Connect 2015 - SoftLayer - Bluemix

With different challenges emerging every other day, startups nowadays have to innovate and operate rapidly in order to achieve exponential growth in a short period of time. Hence, my friends working in startups always complain about the abuse of 4-letter word “asap”. Every task they receive always come with one requirement: It must be done asap. However, as pointed out in the book Rework by Jason Fried from Basecamp, when everything is expected to be done asap, nothing in fact can be really asap. So, how are startups going to monetize their ideas fast enough?

To answer the question, this year IBM Connect Singapore highlighted two cloud platforms, SoftLayer and Bluemix, which help to assist startups to build and launch their products at speed.

IBM Connect 2015 at Singapore Resorts World Sentosa
IBM Connect 2015 at Singapore Resorts World Sentosa

SoftLayer, IaaS from IBM

SoftLayer is a very well-known IaaS cloud service provider from IBM. Currently, SoftLayer has data centres across Asia, Australia, Europe, Brazil, and United States. William Lim, APAC Channel Development Manager at SoftLayer, stated during the event that there will be two new data centres are introduced for every two months on average. In addition, each data centre is connected to the Global Private Network which enable startups to deploy and manage their business applications worldwide.

With Global Private Network, SoftLayer users won’t be charged for any bandwidth usage across the network. Yup, free! Bandwidth between servers on the Global Private Network is unmetered and free. So, with this exciting feature, startups are now able to build true disaster recovery solutuion which requires file transfer from one server to another.

William Lim sharing story about Global Private Network.
William Lim sharing story about Global Private Network.

What excites me during the event is the concept of Bare Metal Server. With Microsoft Azure and Amazon Web Services (AWS), users do not get predictable and consistent performance especially for I/O intensive tasks when their applications are running on virtual-machine based hosting. In order to handle I/O intensive workloads, IBM SoftLayer offers their users a new type of server, Bare Metal Server.

A Bare Metal Server is a physical server which is fully dedicated to one single user. Bare Metal Server can be setup with cutting-edge Intel server-grade processors which can then maximize the server processing power. Hence, for those startups that would like to build Big Data applications, they can make use of Bare Metal Server from SoftLayer to perform data-intensive functions without worrying about latency and overhead delays.

Bluemix, PaaS from IBM

As a user of Microsoft Azure Cloud Service (PaaS), I am very glad to see the Bluemix, PaaS developed by IBM, is also being introduced in the IBM Connect event.

Amelia Johasky, IBM Cloud Leader (ASEAN), sharing how Bluemix works together with three key open compute technologies: Cloud Foundry, Docker, and OpenStack.
Amelia Johasky, IBM Cloud Leader (ASEAN), sharing how Bluemix works together with three key open compute technologies: Cloud Foundry, Docker, and OpenStack.

One of the reasons why I prefer PaaS over IaaS is because in a startup environment, developers always have too many todos and too little time. Hence, it is not a good idea to add the burden of managing servers to the developers. Instead, developers should just focus on innovation and development. In the world of PaaS, tons of useful libraries are made available and packaged nicely which allows developers to code, test, and deploy easily without worrying too much about the server configuration, database administration, and load balancing. (You can read about my pain of hosting web applications on Azure IaaS virtual machines here.)

After the IBM Connect event, I decide to try out Bluemix to see how it’s different from Azure Cloud Service.

The registration process is pretty straightforward. I started with the Web Application Template. In Bluemix, there are many programming languages supported, including the latest ASP .NET 5, the new open-source and cross-platform framework from Microsoft team!

Many web development platforms are available on Bluemix!
Many web development platforms are available on Bluemix!

I like how Bluemix is integrated with Git. It allows us to create a hosted Git repository that deploys to Bluemix automatically. The entire Git setup process is also very simple with just one click of the “Git” button. So every time after I push my commits to the repository, my app will be automatically updated on the server as well. Cool!

Bluemix enables us to deploy our web apps with Git.
Bluemix enables us to deploy our web apps with Git.

You can click on the button below to try out my simple YouTube related web app deployed on Bluemix.

Try out my app hosted on Bluemix at http://youtube-replayer.mybluemix.net/.

Bluemix is underlined by three key open compute technologies, i.e. Cloud Foundry, Docker, and OpenStack. What I have played with is just the Cloud Foundry part. In Bluemix, there is also an option to enable developers to deploy virtual machines. However, this option is currently beta and users can only have access to it if they are invited by IBM. Hence, I haven’t tried their VM option.

Finally, Bluemix currently only offers two regions, UK and US South. So for those who would like to have their apps hosted in other parts of the world, it may not be a good time to use Bluemix now.

YouTube RePlayer is now hosted on Bluemix.
YouTube RePlayer is now hosted on Bluemix.

Azure Blob Storage and File API

Azure Blob Storage - Azure SDK - ASP .NET MVC - Entity Framework - HTML5

When my applications were hosted on Windows Azure Virtual Machines (VM), we stored the images uploaded via our web applications in the hard disks of the VMs (except the temporary disk). However, when we started load balancing, we soon encountered a problem that the uploaded images were only found in one of the VMs. So we needed to find a centralized storage for those images.

Recently, when we are using Azure PaaS (aka Cloud Service), even without load balancing, we already encounter the same issue. That is simply because the hard drives used in Cloud Service instances are not persistent. Hence, a persistent file storage on the cloud is needed.

IaaS vs. PaaS
IaaS vs. PaaS

Blob Storage

Azure Blob Storage, according to Azure Documentation, is a service for storing large amount of unstructured data that can be accessed everywhere via HTTP or HTTPS. Hence, it is an ideal tool that we can use as the persistent image cloud storage.

There are two types of blob, Page Blob and Block Blob. Page Blob is commonly used for storing VHD files for VMs because it is optimized for random read and write operations.

For most of the files uploaded, it’s recommended to store as Block Blobs because large files will be split into smaller blocks and then uploaded concurrently. Hence, Block Blob is designed to give us faster upload and better throughput, which is great for image upload.

The maximum size for a Block Blob is 64 MB. Hence, if the uploaded file is more than 64 MB, we must upload it as a set of blocks; otherwise, we will receive status code 413 (Request Entity Too Large). For my web applications, there is no need for uploading an image which is more than 5MB most of the time. Hence, I can just limit the size of images before the user uploads them.

HttpPostedFileBase imageUpload;
...
if (imageUpload.ContentLength > 0 && imageUpload.ContentLength <= 5242880)
{
    //warn the user to resize the image
}

Let’s Try Uploading Images

I’m going to share how to upload more than one image to the Azure Blob Storage from an ASP .NET MVC 5 application. If you are going to upload just one image, simply remove the for loop and change List to just DBPhoto in the codes below.

First of all, I create a class to handle upload to Azure Storage operation.

public class AzureStorage
{
    public static async Task UploadAndSaveBlobAsync(
        HttpPostedFileBase imageFile, CloudBlobContainer container)
    {
        string blobName = Guid.NewGuid().ToString() + 
            Path.GetExtension(imageFile.FileName);

        CloudBlockBlob imageBlob = container.GetBlockBlobReference(blobName);
        using (var fileStream = imageFile.InputStream) 
        {
            await imageBlob.UploadFromStreamAsync(fileStream);
        }

        return imageBlob;
    }
}

So, in my controller, I have the following piece of code which will be called when an image is submitted via web page.

[HttpPost]
[ValidateAntiForgeryToken]
public async Task Create(
    [Bind(Include = "ImageUpload")] PhotoViewModel model)
{
    var validImageTypes = new string[] { "image/jpeg", "image/pjpeg", "image/png" };
    
    if (ModelState.IsValid) 
    {
        if (model.ImageUpload != null && model.ImageUpload.Count() > 0)
        {
            var storageAccount = CloudStorageAccount.Parse 
                (WebConfigurationManager.AppSettings["StorageConnectionString"]);

            var blobClient = storageAccount.CreateCloudBlobClient();
            blobClient.DefaultRequestOptions.RetryPolicy = 
                new LinearRetry(TimeSpan.FromSeconds(3), 3);  

            var imagesBlobContainer = blobClient.GetContainerReference("images");
            foreach (var item in model.ImageUpload) 
            { 
                if (item != null) {
                    continue;
                }
                
                if (validImageTypes.Contains(item.ContentType) && 
                    item.ContentLength > 0 && item.ContentLength <= 5242880)
                {
                    var blob = await AzureStorage.UploadAndSaveBlobAsync(item, imagesBlobContainer);
                    DBPhoto newPhoto = new DBPhoto(); 
                    newPhoto.URL = blob.Uri.ToString();
                    db.DBPhoto.Add(newPhoto); 
                } 
                else 
                {
                    // Show user error message 
                    return View(model); 
                }
            }
            db.SaveChanges();
            ... 
        } 
        else
        {
            // No image to upload
        } 
    }
    return View(model);
}

In the code above, there are many new cool things.

Firstly, it is the connection string to Azure Blob Storage, which I store in StorageConnectionString in web.config. The format for secure connection string is as follows.

DefaultEndpointsProtocol=https;AccountName=;AccountKey=;
Retrieve the access keys to the Storage Account.
Retrieve the access keys to the Storage Account.

Secondly, it’s LinearRetry. It is basically a retry policy which states how many times the program will retry and how much time needed between retries. In my case, it will only wait for 3 seconds after each try up to 3 tries.

Thirdly, I get the URL of the image on the Azure Blob Storage via blob.Uri.ToString() and store it into the database table. The URL will be used later for displaying the image as well as deleting the image.

Fourthly, I actually check to see if model.ImageUpload has null entries. This is because if I submit the form without any image to upload, model.ImageUpload has one entry. Not zero, but one. The only one entry is actually null. So if I don’t check to see whether the entry in model.ImageUpload is null, there will be an exception thrown.

The controller has such a long code. Luckily the code needed in the model and view is short and simple.

For the model PhotoViewModel, I have the following.

public class PhotoViewModel
{
    ...
    
    [Display(Name = "Current Images")]
    public List AvailablePhotos { get; set; }
}

For view, it is easy to allow selecting multiple files in the same view page. The “multiple = “true”” is to make sure more than one file can be selected in the File Explorer. You can omit this attribute if you only want at most one file being selected.

@Html.LabelFor(model => model.ImageUpload, new { style = "font-weight: bold;" })
@Html.TextBoxFor(model => model.ImageUpload, new { type = "file", multiple = "true" })
@Html.ValidationMessageFor(model => model.ImageUpload)

Image Size and HttpException

The image upload function looks fine. However, when images having size larger than a certain size is uploaded, HttpException will be thrown.

There is no way that having exception would be fun too! (Image Credit: Tari Tari)
There is no way that having exception would be fun too! (Image Credit: Tari Tari)

In order to prevent DOS attacks which upload huge files to the server, IIS by default only allows files which have size less than 4MB to be uploaded. Hence, although I earlier put a check to prevent image larger than 5MB to be uploaded, the exception will still be thrown if an image of size between 4 to 5MB is uploaded.

What if we just change the if clause above to allow only at most 4MB of image being uploaded? This won’t work because the exception is already thrown before the if condition is reached.

Then, can we just increase the IIS limit from 4MB to, let’s say, 100MB or something bigger? Sure. This can work. However, it still doesn’t stop someone uploads something bigger than the limit. Also, it makes attackers easier to exhaust your server with big files. Hence, expanding the upload size restriction is not really a full solution.

If you are interested, there are many good articles online discussing about this problem. I highlight some interesting ones below.

  1. Use HttpModule to Handle File Uploads;
  2. Use RIA (Rich Internet Application) Services in Silverlight (Seriously, we are talking about Silverlight in year 2015?);
  3. SubStatusCode = 13 in IIS 7;
  4. Catch the Exception in Global.asax.

I don’t really like the methods listed above, especially the 3rd and 4th options. It’s already too late to inform the user when the exception is thrown. Could we do something at client side before the images are being uploaded?

Luckily, we have File API in HTML 5. It allows to loop through the files in JavaScript to check their size. So, after the submit button is clicked, I will call a JavaScript method to check for the size of the images before they are being uploaded.

function IsFileSizeAcceptable() {
    if (typeof FileReader !== "undefined") {
        var filesBeingUploaded = document.getElementById('ImageUpload').files;
        for (var i = 0; i < filesBeingUploaded.length; i++) {
            if (filesBeingUploaded[i].size >= 4194304) { // Less than 4MB only
                alert('The file ' + filesBeingUploaded[i].name + ' is too large. Please remove it from your selection.');
                return false;
            }
        }
    }
    return true;
}
File API is currently supported in major modern browsers. (Image Credit: http://caniuse.com/#feat=fileapi)
File API is currently supported in major modern browsers. (Image Credit: http://caniuse.com/#feat=fileapi)

Remove from Azure Blob Storage

It’s normal that files uploaded to storage will be removed later. So how are we going to implement this feature in our ASP .NET MVC 5 application?

First of all, I added the following code to my AzureStorage.cs.

public static async Task DeleteBlobAsync(Uri blobUri, CloudBlobContainer container)
{
    string blobName = blobUri.Segments[blobUri.Segments.Length - 1];
    CloudBlockBlob blobToDelete = container.GetBlockBlobReference(blobName);

    await blobToDelete.DeleteAsync(); 
}

Secondly, I just pass in the Azure Storage URL of the image that I would like to remove and then call the DeleteBlobAsync method.

Uri blobUri = new Uri();
await AzureStorage.DeleteBlobAsync(blobUri, imagesBlobContainer);

Then the image will be deleted from the Azure Storage successfully.

Global.asax.cs and Blob Container

In order to have my application to create a blob container automatically if it doesn’t already exist, I add a few lines in Global.asax.cs as follows.

var storageAccount = CloudStorageAccount.Parse(
    WebConfigurationManager.AppSettings["StorageConnectionString"]);
var blobClient = storageAccount.CreateCloudBlobClient();
var imagesBlobContainer = blobClient.GetContainerReference("images");
if (imagesBlobContainer.CreateIfNotExists())
{
    imagesBlobContainer.SetPermissions(new BlobContainerPermissions
        {
            PublicAccess = BlobContainerPublicAccessType.Blob
        });
}

Write a Console Program to Upload File to Azure Storage

So, how is it done if we are developing a console application, instead of web application?

Windows Azure Storage NuGet Package needs to be installed first.
Windows Azure Storage NuGet Package needs to be installed first.

The codes below show how I upload an html file from my local hard disk to Azure Blob Storage. Then I can share the Azure Storage URL of the file to my friends so that they can read the web page.

Similar to what I do in web application, this is how I connect to the Storage account via https.

var azureStorageAccount = new CloudStorageAccount(
    new StorageCredentials("", ""), true);

This is how I access the container.

var blobClient = new CloudBlobClient(azureStorageAccount.BlobStorageUri, azureStorageAccount.Credentials);
var container = blobClient.GetContainerReference("myfiles");

Then the next thing I do is just upload the local file to Azure Storage by specifying the file name, content type, etc.

CloudBlockBlob blob = container.GetBlockBlobReference("mysimplepage.html");
using (Stream file = System.IO.File.OpenRead(@"C:\Users\ChunLin\Documents\mysimplepage.html")) 
{
    blob.Properties.ContentType = "text/html"; 
    blob.UploadFromStream(file); 
}

Yup, that’s all. =)

Pricing

Hosting your files on cloud storage is sure convenience. However, Azure Blob Storage is not free. The following table shows the current pricing of Azure Block Blob Storage in South East Asia region. To get the latest pricing details, please visit Azure Storage Pricing page.

Azure Standard Block Blob Storage in SEA Pricing
Azure Standard Block Blob Storage in SEA Pricing

Summer 2015 Self-Learning Project

This article is part of my Self-Learning in this summer. To read the other topics in this project, please click here to visit the project overview page.

Summer Self-Learning Banner