Machine Learning based Password Strength Classification

Machine Learning based Password Strength Classification


Lately, I have been wanting to do a few projects on Machine Learning and Cyber Security. This is another project related to infosec and machine learning.

Most of the password strength meters vary from company to company. All these strength meters are rule based. A password can be strong for google while being weak for dropbox. I was thinking what if we can let a machine learning algorithm decide whether our passwords are weak, normal or strong. This idea lead me to this post. Lets dive right in.

Machine Learning based Password strength checking

Data Collection:

The passwords used in our analysis are from 000webhost leak that is available online. How did we figure out which passwords were stronger and which were weaker? Well, there is a tool called PARS by Georgia Tech university which have all the commercial password meters integrated into it. All I did was give that tool all the passwords and it gave me new files for each commercial password strength meter. The files contained the passwords with one more column i.e their strength based on the commercial password strength meters.

The commercial password strength algorithms I used are of Twitter, Microsoft and battle. How is this algorithm different from these strength meters? First of all, it is entirely based on machine learning rather than on rules. Secondly, I only kept those passwords that were flagged weak, medium and strong by all three strength meters. This means that all the passwords were indeed either weak, medium or strong.

Here are a few passwords that are classified using commercial password strength algorithms.

I had a total of 3 million passwords but after taking the intersection of all classifications of commercial meters, I was left with 0.7 million passwords which I then split into test(20%) and training(80%) set. The reduction was because of the fact that I only used passwords that were flagged in a particular category by all three algorithms.

Analysis:

Lets dive into the code.

I’ll be using Tf-idf scores but instead of using the whole password, I’ll be using each character as a token. Other metrics that I have not used could be password lengths, number of special characters used, number of digits used and so on. Here’s the custom tokenizer.

Loading data.

Lets shuffle the data and get our y and X vectors.

Now, lets apply the Tfidvectorizer on our corpus of passwords and split the data.

That’s it. All we have to do is apply our machine learning algorithm and we have a machine learning based password strength checker. Lets do it. I used logistic regression with multi class classification since I wanted the algorithm to run fast.

Our accuracy comes out to be 81% which is great considering we did not use a large amount of data. This means that 80% of the time, our algorithm classifies the passwords would have been jointly classified by three commercial password classifiers. Lets check how this algorithm classifies our passwords.

Lets look at the result.

Looks like our algorithm knows that having a combination of letters, numbers and special characters is a good thing. Great, the results look promising, don’t they?

A few things to note. The algorithm learnt the rules from existing algorithms but I combined the results of multiple algorithms to make it robust. It is not just imitating a rule based algorithm. It is imitating an amalgam of many password strength checking algorithms used in the wild. This is a project that I did in my spare time and is by no means a comprehensive one. It was an idea that I wanted to execute and see the results and share them with others. I believe that this can be further extended into much more meaningful and a proper strength checker/classifier.

This was a fun project. Comments and suggestions are most welcome.

The code and data can be found at Github.

 

2 Comments

Add yours
  1. 1
    Bilge Gunduz

    When you mentioned the idea lead you you wrote “this post”. But there is no link to attached to it. You may forgot. By the way, i guess i didn’t get it what is the difference between your project and the PARS project.

+ Leave a Comment