Introduction

This blog post is part of the final project for the Udacity Machine Learning Engineer Nanodegree Program using AWS SageMaker

The project contains simulated data that mimics customer behavior on the Starbucks rewards mobile app. The goal of the project is to analyze this "historical" data in order to implement an algorithm that finds the most suiting offer type for each customer.

The data is contained in three files:

• portfolio.json - containing offer ids and meta data about each offer (duration, type, etc.)
• profile.json - demographic data for each customer
• transcript.json - records for transactions, offers received, offers viewed, and offers completed

Here is the schema and explanation of each variable in the files:

portfolio.json

• id (string) - offer id
• offer_type (string) - type of offer ie BOGO, discount, informational
• difficulty (int) - minimum required spend to complete an offer
• reward (int) - reward given for completing an offer
• duration (int) - time for offer to be open, in days
• channels (list of strings)

profile.json

• age (int) - age of the customer
• became_member_on (int) - date when customer created an app account
• gender (str) - gender of the customer (note some entries contain 'O' for other rather than M or F)
• id (str) - customer id
• income (float) - customer's income

transcript.json

• event (str) - record description (ie transaction, offer received, offer viewed, etc.)
• person (str) - customer id
• time (int) - time in hours since start of test. The data begins at time t=0
• value - (dict of strings) - either an offer id or transaction amount depending on the record

Solution Statement

This is a classification problem, my approach is going to be: create a Machine learning model to
predict the best offer for the user, for BOGO or discount (we are going to leave out the
informational offers, which have no real "conversion")
There are the steps to follow:

• Fetching data
• Data Cleaning
• Data Preparation
• Data visualization and analysis
• Train Model
• Evaluate the model

Evaluation Metrics

Since this is a classification problem we can use the following metrics to evaluate the model:

• Precision The proportion of positive cases that were correctly identified.
• Recall The proportion of actual positive cases which are correctly identified.
• F1-score, that combines the two previous measures.
• Roc_auc_score Area Under the ROC Curve (AUC), a measure that calculates the area
  under the Receiving Operating Characteristic Curve. This particular curve accounts that
  higher probabilities are associated with true positives and vice-versa.

Algorithms evaluated

We explored two algorithms Linear Learner and XGBoost to find the best models for for each offer, a Discount, a Buy One Get One (BOGO)
or Informational.

Project design

During the fetching data process, we have to join all the different pieces of information coming
from the 3 data sources, we need to fetch the data.

Data preparation. After analyzing the data, we transform the dataset through different stages
missing imputation, categories encoding, data standardization, etc.

Clean and Visualize the data to understand it's content and analyze it, to find possibles outliers
and remove them (if possible).

Data distribution visualization

Feature Engineering

Preprocessing with scikit-learn

• Encode categorical variable gender (One Hot encoder)
• Impute missing values
• Normalize feature distributions

Things to notice:

• There are 3 genre recorded in the dataset (F, M, O) which is going to be encoded during the feature engineering phase.
• The scatterplot shows some odd structure but it should be because the data was simulated, also it doesn't look to be outliers.

Train the model

We used different ML models, one to predict the BOGO, the other to
predict the Discount and another to predict the informational offer. For each model, we try two algorithms, Linear Learner and Amazon Sage maker XG-Boost built in Algorithm.

For each algorithm, we tune the hyper-parameters to find the one with the best performance.
With the best two models, we can combine the results in order to obtain a single type of offer for
the user.

Evaluate the model we measure and compare the performances of the models with the
current benchmark, to learn if the proposed solution is viable for the offer attribution process.

Notebooks

The 3 notebooks and the data used for this project can be found in the following github repository:
https://github.com/jomendez/starbucks-captone-machine-learning.git

• data-exploration.ipynb
• feature_engineering.ipynb
• training_the_model.ipynb

Results

Simplistic CRUD ionic 5 firestore TODO app example

This is a very simplistic CRUD (Create, Read, Update, Delete) Created with ionic 5 and firestore (database).

You can download the source code here.

Note:
I used this design as inspiration: https://dribbble.com/shots/4782162-To-Do-App-Concept-Minimal.

The idea here is to illustrate how to create a simple CRUD with firestore.

Installation.

My advice is that you install nvm (to manage your node versions), here is tutorial step by step.

After you install nvm run the following commands:

nvm install 12

If everything went well you'll have installed node 12.0.0 .

Now you'll need to install ionic cli:

npm install -g @ionic/cli

Now is time to download the project and install it's dependencies.

git clone https://github.com/jomendez/ionic-5-crud-todo-with-firestore.git
cd ionic-5-crud-todo-with-firestore
npm install

Next step it to create a config file that will contain your firebase config keys here:

src/app/config/firebase.ts

If you are on bash you can use the following command:

touch src/app/config/firebase.ts

Assuming that you are familiar with firebase and already have an account, paste your configuration info into the firebase.ts file, it should looks something like this (This is just and example), If you are not familiar with firebase or doesn't have an account, I'll provide more information on how to do it bellow.

Example of configuration:

export const firebaseConfig = {
  apiKey: "AIzaSwERfdilPYtcNDfvFR6x944EowerdfghfSDFsfg",
  authDomain: "crud-456657.firebaseapp.com",
  databaseURL: "https://crud-456657.firebaseio.com",
  projectId: "crud-456657",
  storageBucket: "crud-456657.appspot.com",
  messagingSenderId: "10130122343423",
  appId: "1:1013012342277:web:sd9f8sd8fs8dfs89df89s",
  measurementId: "G-3FGJ34GFGJK"
};

How to get your firebase configuration.

Go to https://firebase.google.com/ and login with your account or create a new one.

Click on create a web app (This is the screen that you'll see the first time you create a projects):

Or add a new project (If you have already other projects created):

Enter the name of your project, accept the terms, and finish the other two steps on the wizard:

Next you'll see something like this, from here you can grab the configuration key.

If you missed the previous screen you can always go to the cog wheel on the left side panel, and click on project settings.

Now scroll down to the Your App section and click on config radio button, and copy the code highlighted on the image bellow, and paste it on the firebase.ts file.

About the author

I describe myself as a passionate Front-end developer, focusing on create, test, and optimize full web apps with compelling user interfaces that runs in web browsers on any device, from desktop computer to tablets and cellphones, therefore, my principal allies are javascript, html, css, and the browser's event loop. That's why you will always find me learning and improving my skills to master those pillars and it's surrounding frameworks and technologies, such as angular, nodejs, ionic, react, etc.

Goals

Some times I find it a little bit hard to interpolate/transmit the description above to recruiters and hiring managers, just with my online presence (before any interview or conversation). That's why my intention is that they just get it by seeing and interacting with this portfolio.

Thus I decided to create a portfolio/profile/interactive resume as a complement for my actual resume and Linkedin profile, in addition to that, also for this two reasons:

1. To challenge my skills with css animations and interaction.

2. To create a showcase and challenge myself in a cool framework that I was recently immersing: Threejs which is used to create 3d environments, interactions, etc. on the web.

I wanted to do something different for this project, so I thought... why don't I create a 3d interactive version of myself. I did a research on the internet and I found this article, about how to create an interactive 3D character with Threejs, and I used it as a guide to create my own 3d interactive character:

https://tympanus.net/codrops/2019/10/14/how-to-create-an-interactive-3d-character-with-three-js/

About the (portfolio/profile/interactive resume)

Now, the portfolio/profile/interactive resume can be broken down into 2 main parts:

1. The get to know the non-technical side of the person, the activities that he likes more (the 4 inner buttons) click on the buttons to know more about the author and see the interactions and animations of the character...

2. The more technical part, more like a resume, showing the technical skills, places he worked, some of his recent work (games, Ionic starters, etc.), featured blog posts, certifications etc.That you can access by clicking on the outer buttons, (placed in the corners)

Portfolio

You can navigate to the portfolio/profile/interactive resume here:

http://www.jomendez.com/portfolio/

javascript Proxy.

This a new feature introduced in the ES6 standard. The Proxy object is used to define custom behavior for fundamental operations (e.g. property lookup, assignment, enumeration, function invocation, etc).

The basics.

There are 3 key terms we need to define before we proceed:

• handler — the placeholder object which contains the trap(s).
• traps — the method(s) that provide property access.
• target — object which the proxy virtualizes.

Here is a list of available Traps.

• apply
• construct
• defineProperty
• deleteProperty
• get
• getOwnPropertyDescriptor
• getPrototypeOf
• has
• isExtensible
• ownKeys
• preventExtensions
• set
• setPrototypeOf

The list going on and on...

The syntax

let proxy = new Proxy( target, trapObject );

Quick example.

Let's see a quick example before we dive in into create our array query library.

class Student {
    constructor(first, last, scores) {
        this.firstName = first;
        this.lastName = last;
        this.testScores = scores;
    }
    get average() {
        let average = this.testScores.reduce( 
            (a,b) => a + b, 
            0 
        ) / this.testScores.length;
        return average;
    }
}

//instantiate the student class  
let john = new Student( 'John', 'Doe', [70, 80, 90] );

let johnProxy = new Proxy( john, {
   //we trap the get and print the property accessed 
    get: function( target, key, context ) {
        console.log( `john[${key}] was accessed.` );
        return john[key];
    } 
});

johnProxy.firstName;
//output.
//> john[firstName] was accessed.
//> <property content>

In this example we are just intercepting the properties called, for example johnProxy.firstName and printed it to the console, this could be used for debugging purpose in dev environments. We can do the same and intersect the 'set' and print a message to the console when we set the value on a property.

Let's create our query library.

It would be similar to write an assertion library, where you can use the name of the field you want to query withing the method's call instead of passing it as a parameter.

First let's create the assertion object.

Here we are going to define the assertion operations, our proxy object will return the objects based on this conditions:

let assertions = {
  Equals: (object, value) => object === value,
  IsNull: (object, value) => object === null,
  IsUndefined: (object, value) => object === undefined,
  IsEmpty: (object, value) => object.length === 0,
  Includes: (object, value) => object.includes(value),
  IsLowerThan: (object, value) => object < value,
  IsGreaterThan: (object, value) => object > value,
  EndsWith: (object, value) => object.endsWith(value),
  StartsWith: (object, value) => object.startsWith(value),
}

The proxy object.

Here we going to trap the get, of the target object to read the name of the property and make sure it starts with findWhere then we use the object field name we want to query, then the assertion (from the assertions above), So the syntax will looks something like this arr.findWhereNameStartsWith('Jo')

let wrap = arr => {
  const prefix = 'findWhere';
  let assertionNames = Object.keys(assertions);
  return new Proxy(arr, {
    get(target, propKey) {
      if (propKey in target) return target[propKey];
      var assertionName = assertionNames.find(assertion =>
        propKey.endsWith(assertion));
      if (propKey.startsWith(prefix)) {
        var field = propKey.substring(prefix.length,
          propKey.length - assertionName.length).toLowerCase();

        var assertion = assertions[assertionName];
        return value => {
          return target.filter(item => assertion(item[field], value));
        }
      }
    }
  })
}

Declare the object.

Next step is to declare the object and wrap it up with the proxy.

let arr = wrap([
  { name: 'John', age: 23, skills: ['mongodb', 'javascript'] },
  { name: 'Lily', age: 20, skills: ['redis'] },
  { name: 'Iris', age: 43, skills: ['python', 'javascript'] }
]);

Calling the methods

console.log(arr.findWhereNameEquals('Lily')) // finds Lily.
console.log(arr.findWhereSkillsIncludes('javascript')) // finds Iris.

The code above the first line will return the object where Property Name is equal to Lily and print it to the console, and the second line will return the object(s) where skills property includes javascript within the array.

Full source code.

Use the following code to play more with javascript proxy.

Things to notice.

Notice how we call the a method in the array that we didn't declared before arr.findWhereAgeIsLowerThan(30) we use the proxy to read this method, the expected syntax is findWhere + <array_field_name> + <assertion> this allow us to create custom queries using the array field name we want to query within the method name:

• arr.findWhereAgeIsLowerThan(27);
• arr.findWhereNameEquals('Lily');
• arr.findWhereSkillsIncludes('javascript');

Wrapping up

With JavaScript proxy, you can wrap up an existing object and intercept any access to its attributes or methods. Even if they don't exist!

There are many real-world applications for Proxies:

• Create SDK for an API.
• validation.
• value correction.
• Revoke access to an object property/method.
• Querying arrays with more readable methods (the main topic of this post).
• property lookup extensions.
• tracing property accesses.
• Monitoring async functions.

Conclusions.

Javascript proxy enables a lot of possibilities to create more readable APIs where you can combine field names, assertions and create dynamics methods on the fly, however as you can see in the link below IE doesn't support it and this can be a downside to create code that needs to be supported by all major browsers. Unfortunately, it’s not possible to polyfill or transpile ES6 proxy code using tools such as Babel, because proxies have no ES5 equivalent.

For more info about javascript proxy and browser compatibility you can Learn more here