What features would you use to predict the Uber ETA for ride requests?

Data processing

Feature Selection

Model Selection

Cross Validation

Evaluation Metrics

Testing and Roll Out

How would you evaluate the predictions of an Uber ETA model?

Data processing

Feature Selection

Model Selection

Cross Validation

Evaluation Metrics

Testing and Roll Out

Describe how you would build a model to predict Uber ETAs after a rider requests a ride.

Data processing

Feature Selection

Model Selection

Cross Validation

Evaluation Metrics

Testing and Roll Out

Suppose you're working as a data scientist at Facebook. How would you measure the success of private stories on Instagram, where only certain chosen friends can see the story?

Data processing

Feature Selection

Model Selection

Cross Validation

Evaluation Metrics

Testing and Roll Out

Precision vs Accuracy Vs Recall?

Accuracy - Accuracy is the most intuitive performance measure and it is simply a ratio of correctly predicted observation to the total observations. One may think that, if we have high accuracy then our model is best. Yes, accuracy is a great measure but only when you have symmetric datasets where values of false positive and false negatives are almost same. Therefore, you have to look at other parameters to evaluate the performance of your model. For our model, we have got 0.803 which means our model is approx. 80% accurate.

Accuracy = TP+TN/TP+FP+FN+TN

Precision - Precision is the ratio of correctly predicted positive observations to the total predicted positive observations. The question that this metric answer is of all passengers that labeled as survived, how many actually survived? High precision relates to the low false positive rate. We have got 0.788 precision which is pretty good.

Precision = TP/TP+FP

Recall (Sensitivity) - Recall is the ratio of correctly predicted positive observations to the all observations in actual class - yes. The question recall answers is: Of all the passengers that truly survived, how many did we label? We have got recall of 0.631 which is good for this model as it's above 0.5.

Recall = TP/TP+FN

F1 score - F1 Score is the weighted average of Precision and Recall. Therefore, this score takes both false positives and false negatives into account. Intuitively it is not as easy to understand as accuracy, but F1 is usually more useful than accuracy, especially if you have an uneven class distribution. Accuracy works best if false positives and false negatives have similar cost. If the cost of false positives and false negatives are very different, it's better to look at both Precision and Recall. In our case, F1 score is 0.701.

F1 Score = 2(Recall Precision) / (Recall + Precision)

Error vs variance vs bias?

In statistics and machine learning, the bias-variance tradeoff is the property of a set of predictive models whereby models with a lower bias in parameter estimation have a higher variance of the parameter estimates across samples, and vice versa. The bias-variance dilemma or problem is the conflict in trying to simultaneously minimize these two sources of error that prevent supervised learning algorithms from generalizing beyond their training set: The bias is an error from erroneous assumptions in the learning algorithm. High bias can cause an algorithm to miss the relevant relations between features and target outputs (underfitting). The variance is an error from sensitivity to small fluctuations in the training set. High variance can cause an algorithm to model the random noise in the training data, rather than the intended outputs (overfitting). The bias-variance decomposition is a way of analyzing a learning algorithm's expected generalization error with respect to a particular problem as a sum of threeterms, the bias, variance, and a quantity called the irreducible error, resulting from noise in the problem itself.

False negatives vs false positives? When is either one worse than the other?

False positive: A false positive error, or in short a false positive, commonly called a ""false alarm"", is a result that indicates a given condition exists, when it does not. For example, in the case of ""The Boy Who Cried Wolf"", the condition tested for was ""is there a wolf near the herd?""; the shepherd at first wrongly indicated there was one, by calling ""Wolf, wolf!"" A false positive error is a type I error where the test is checking a single condition, and wrongly gives an affirmative (positive) decision. However it is important to distinguish between the type 1 error rate and the probability of a positive result being false. The latter is known as the false positive risk (see Ambiguity in the definition of false positive rate, below).

False negative: A false negative error, or in short a false negative, is a test result that indicates that a condition does not hold, while in fact it does. In other words, erroneously, no effect has been inferred. An example for a false negative is a test indicating that a woman is not pregnant whereas she is actually pregnant. Another example is a truly guilty prisoner who is acquitted of a crime. The condition ""the prisoner is guilty"" holds (the prisoner is indeed guilty). But the test (a trial in a court of law) failed to realize this condition, and wrongly decided that the prisoner was not guilty, falsely concluding a negative about the condition. It depends, must answer why it depends.

Describe your data science process start to finish?

ID: Identify question, identify best data, identify metrics or features. LOAD: separate training and testing, load data and clean it, perform exploratory analysis, Fit: models (start simple unless yo know it won't work) CV to train hyper parameters. Find best model and run evaluation set

Data science vs machine learning vs AI?

Data Science: Data science is the extraction of relevant insights from data. It uses various techniques from many fields like mathematics, machine learning, computer programming, statistical modeling, data engineering and visualization, pattern recognition and learning, uncertainty modeling, data warehousing, and cloud computing. Data Science does not necessarily involve big data, but the fact that data is scaling up makes big data an important aspect of data science.

Machine Learning: Machine learning is the ability of a computer system to learn from the environment and improve itself from experience without the need for any explicit programming. Machine learning focuses on enabling algorithms to learn from the data provided, gather insights and make predictions on previously unanalyzed data using the information gathered. Machine learning can be performed using multiple approaches. The three basic models of machine learning are supervised, unsupervised and reinforcement learning.

Artificial Intelligence: Artificial intelligence refers to the simulation of a human brain function by machines. This is achieved by creating an artificial neural network that can show human intelligence. The primary human functions that an AI machine performs include logical reasoning, learning and self-correction. Artificial intelligence is a wide field with many applications but it also one of the most complicated technology to work on. Machines inherently are not smart and to make them so, we need a lot of computing power and data to empower them to simulate human thinking.

How would you find correlation between a categorical variable and a continuous variable?

Distance Metrics: Although the concept of "distance" is often not synonymous with "correlation," distance metrics can nevertheless be used to compute the similarity between vectors, which is conceptually similar to other measures of correlation. There are many other distance metrics, and my intent here is less to introduce you to all the different ways in which distance between two points can be calculated, and more to introduce the general notion of distance metrics as an approach to measure similarity or correlation. I have noted ten commonly used distance metrics below for this purpose.

Contingency Table Analysis: When comparing two categorical variables, by counting the frequencies of the categories we can easily convert the original vectors into contingency tables. For example, imagine you wanted to see if there is a correlation between being a man and getting a science grant (unfortunately, there is a correlation but that's a matter for another day). Your data might have two columns in this case — one for gender which would be Male or Female (assume a binary world for this case) and another for grant (Yes or No). We could take the data from these columns and represent it as a cross tabulation by calculating the pair-wise frequencies.

How do you treat null/missing values? Name 3 methodologies.

Last Observation Carried Forward (LOCF) & Next Observation Carried Backward (NOCB)

This is a common statistical approach to the analysis of longitudinal repeated measures data where some follow-up observations may be missing. Longitudinal data track the same sample at different points in time. Both these methods can introduce bias in analysis and perform poorly when data has a visible trend

Linear Interpolation

This method works well for a time series with some trend but is not suitable for seasonal data

Seasonal Adjustment + Linear Interpolation

This method works well for data with both trend and seasonality

How can outlier values be treated?

Univariate outliers: Box plots, histograms, distributions

Multivariate method: Fit several miltivariate models and view if some data points are just off parht without consistency.

What is data normalization? Name 2 normalization methodologies.

In databases it is makin sure there is no redundancy and data dependencies are all logical. In data it can be reducing a feature set so that it is based around a mean and std dev, or between the values 0 and 1

What is the role/importance of data cleaning?

Unstrucutred data is totally unusable until cleaned. Data coming from differnet soruces can't be compared until cleaned. Lastly, make sure you emphasize necessity of making defined data cleaning proess so repeateable and able to go into a product

What are success metrics vs tracking metrics?

Metrics: What you measure to gauge performance or progress within a company or organization. Your most important metrics are your key performance indicators, or KPIs.

Analytics: Analytics use metrics to help you make decisions about how to move forward.

What kind of metric would you make to measure success of a program (marketing) and how do you define them?

Measure by categeory, segment metrics based on what they report. Evaluate pros and cons of as many metrics as possible but only end on a few. Try to tie everything back to revenue impact.

Let's say an app was getting a redesign. How do you know if the redesign was successful?

Average page time (without force quit)

Average session duration - (does it go up) (more pages faster?)

Gather Qualitative Feedback from Would-Be Customers

Make Sure you Didn't Make Technical Mistakes

Map and Measure Your Buyers Journeys and Marketing Funnels Using Micro-Conversions

Number of sessions per user (which should go up) - Conversion rate and (New vs Old)

Organic search traffic

Pages per sessions (should also go up) - (set up A/B test before and after) (bounce rate)

Predict & Measure the ROI of Your Website Redesign Efforts

Track The Success of Each New or Updated Website Page

Use Heatmap & Other Advanced Analytics Tools to Further Improve

We noticed a steep decline in users in a certain area of the world, how would you address/asses?

Active users (ADAU, MAU) - ADAU = total number of active users in a month / days in the average month

Conversion rates - Conversion rate = # of users who completed an event / total # of users

Customer satisfaction (CSAT) - NPS = % of promoters - % of detractors

Monthly active users (MAU) is similar to ADAU but is the average number of active users in a given month, averaged over the months in a year. - MAU = total number of active users in a year / 12

Retention rate - Retention rate = (# of customers at end of period - # of customers acquired during period) / # of customers at start of period

Screenflow - Golden path completion = the % of users that complete the golden path screenflow

Session interval - Session interval = various analyses of session lengths

Session length - Session length = various analyses of session lengths

Stickiness - Stickiness = average daily active users (ADAU) / monthly active users (MAU)

Time in-app - Time in-app = the average number of hours users spend in-app per day, week, or month"

Define precision and recall.

Answer: Recall is also known as the true positive rate: the amount of positives your model claims compared to the actual number of positives there are throughout the data. Precision is also known as the positive predictive value, and it is a measure of the amount of accurate positives your model claims compared to the number of positives it actually claims. 

It can be easier to think of recall and precision in the context of a case where you've predicted that there were 10 apples and 5 oranges in a case of 10 apples. You'd have perfect recall (there are actually 10 apples, and you predicted there would be 10) but 66.7% precision because out of the 15 events you predicted, only 10 (the apples) are correct.More reading: Precision and recall (Wikipedia)

 Auto Encoder vs Encoder Decoder

Autoencoders exhibit encoder-decoder structure:

Autoencoders are used in supervised as well as unsupervised learning tasks. They can consist of convolution, dense or even recurrent layers. As the name suggests, they tend to encode their inputs in smaller dimensional space.

They are primarily used for dimensionality reduction and other tasks like image generation, segmentation, neural implanting etc.

What operates better, LMS or Perceptron?

LMS; perceptron converges to a solution to correctly categorize patterns, but its result is prone to noise since patterns are often close to decision boundaries. LMS tries to move the boundaries far from training patterns.

How do you use the Taylor Series against the performance index?

By approximating the p.i., we are limiting the expansion finitely because its derivatives exist.

In a single-neuron perceptron, what weights and biases are changed to distinguish between 2 outputs?

The weights in the weight matrix that differentiate between the target outputs will change, such as if [1 -1 -1] is orange and [1 1 -1] is apple, the weight matrix will be [0 1 0]