Updates on the data breach

Announcements 📰
#1

21 July 2022 Update: Updates about the state of our systems

If you haven’t already, please read this post first.

Hi everyone,

After a hectic few days since our first announcement about the data breach, I wanted to follow up on a few topics that were mentioned here on the forums:

What happened?

As far as we know someone got access to either one of our IPs, or to one of our supplier’s systems and started randomly authorizing transactions on behalf of many of our customers.
There is no indication about any card data being leaked as we do not store identifiable information for debit or credit cards. The format of debit cards that we save on our systems looks a little bit like the following:

src_c64c736871f7a24e

There is nothing connecting this information to a specific person or a specific card. In fact, the ids are completely useless if not used together with our IP, username and password to actually authorize payments on it.

The payments that have been made were refunded and everyone who contacted us got proof of refund PDFs. Some banks take a couple of days to process refunds, so if you were not issued one already, please ping us at support@vitracash.com and we will send you the proof so you can follow up with your bank.

Is this a notifiable breach?

No. There is no indication that any data was stolen that would render this breach notifiable.

Our systems were deliberately put into a malfunctioning state to randomly charge users’ cards.

Why do you not communicate faster/better etc.?

Our top priority was to prevent further bad things from happening. Just as much as answering all support requests, refunding the transactions etc. I appreciate it must be difficult to be in the dark for so long. But until yesterday all of our employees were 24/7 at work preventing even worse things from happening and giving people their money back.
We will try to update you on the progress daily from Monday onwards. I will also stay available for any further questions. I hope those insights will help you assess the situation yourself and understand what we are working on.

What are you planning on doing to prevent those attacks in the future?

We turned off our systems for the time being as we are working on our infrastructure and core security. This will go on for at least another week as we want to make sure everything is done from the ground up with security in mind.

Mid-term we will hire a dedicated IT-security DevOp to help us keep our infrastructure secure while also helping us scale.

For the time being, we drastically reduce our attack surfaces by switching from Kubernetes to DigitalOcean App Platform, moving our databases to the same private subnet and disabling public IPs, whitelisting all communication between our systems and last but not least creating Firewalls for each and every service we deploy that prevents access to unmanaged ports.

I certainly believe that even though this attack was super painful to deal with, it pushed us in the right direction from a security standpoint. This will only make us better in the future.

When can I sign in to the app again?

Probably in 1-2 weeks.

Can I delete my VitraCash user account / connected cards?

Yes, please contact our support at support@vitracash.com

Do I need to freeze / reissue connected cards?

There is no indication that any card data was compromised. Our acquirer’s platform is PCI compliant and there have been no published breaches of their servers recently. We consider it safe to keep your cards and not re-issue them. Though this is up to you of course and extra security never hurt anyone.

How did this / will this affect our public image?

Data breaches happen even to the best. We really worried that this one might be really hard to swallow for a lot of people. Fortunately, we got a lot of great feedback for being so fast at responding and refunding transactions. If we saw anything, it’s that a lot of people actually started to trust us more as we handled this situation with as much professionalism as possible. We had a lot of great advocates defend us on our Telegram Group (without us asking for it).

While I do not want to say there is anything good to this breach in terms of publicity, and there were obviously a lot of angry users as well, I do think that handling bad situations well is by far preferable to not knowing how bad situations will be handled.

I was not refunded

Please contact us at support@vitracash.com

My refund is less than what was charged

This is due to some banks applying FX fees for refunds (don’t even ask). We cannot refund more than the original amount but we will happily either 1) refund your VitraCard physical card shipment of 10£ or offer you a share code so you can order it for free if you haven’t already or 2) wire you the outstanding amount.

In both cases, please contact support@vitracash.com and let us know how much exactly your bank charged you.

To wrap it up: I remain at your disposal. Please ask any questions that are not sensitive here in this thread to keep it organized. If anyone asks questions in other threads, please refer them to this one.

I hope you have a great weekend ahead of you!

Koray.

8 Likes
Updates about the state of our systems
Fraudulent Card Charges - Statement
Fraudulent Card Charges - Statement
#2
#3

Thank you for the update @Koray, very reassuring.

5 Likes
#4

It looks like something crucial is missing here: You clearly cannot rely on the security of an IP for security.

If that’s all that was needed that would be a very very sorry state of affairs. Did they also get your credentials?

6 Likes
#5

This is actually not up to us. One of our key suppliers has this form of security where they simply whitelist IPs that can access their APIs.

But having access to an IP of a product always causes significant problems.
Yes, they had the credentials to authorize payments using the card ids.

1 Like
#6

Seems like a bit of a bigger deal than access to an IP don’t you think?

Poor password hygiene? Inside job? I hope the password wasn’t password123

2 Likes
#7

The unauthorized access to our IP is definitely the bigger issue here. The password is an autogenerated 64-byte API key that cannot be guessed. But using the access they had they could have done a lot of things (which is yet to be confirmed). One of which is reading outgoing packets to our suppliers and extracting the API key.

1 Like
#8

Goodness. Is the api key not encrypted in transit? Access to an ip should never ever allow interception of the api key. That this was possible indicates a far bigger problem than you seem to realise.

Might be worth expediting the employment of a security professional rather than pushing it to ‘medium term’

2 Likes
#9

Thank you for the update

As a non-technical person, can you explain how the move of platform will help improve security?

So someone got access to all 3 elements? Do you not use 2 factor authentication?

This may not be intended, but it reads as suggesting that you didn’t have firewalls in place before?

4 Likes
#10

We are not able to confirm all the details but if you really want to dig deeper you can hit me up and I’ll explain it.
Regarding your point: Having access to the same machine that sends out the requests makes TLS useless. Neither HTTPS nor any other protocol for that matter would withstand this.

1 Like
#11

Ah. So it’s not just ‘access to an ip’. It’s a server compromise

3 Likes
#12

And this is where we come to a point that is neither confirmed nor easy to detect. This is why I am trying not to be red herringed. Please simply accept the fact that we don’t know yet and will release more details in due course instead of putting assumptions on the table.

Maybe they guessed the API key. Maybe they compromised one of our executives. Maybe they compromised our supplier. And yes, maybe it was server access. We simply don’t know and I don’t want to play guessing game.

3 Likes
#13

Yes, sure. Kubernetes is very hard to maintain and manage properly. It has a lot of fine-grained details that need to be looked after constantly. Switching to App Platform means we get a fully managed service where we only need to look after the security of our own code instead of the infrastructure itself.

That’s complicated. APIs typically have API keys as an access method. Because they are pretty long and hard to guess this is mostly the only form of authentication. We had IP whitelisting on top of that, which is generally a good extra. 2 factor is not really a thing with automated systems.

We had Firewalls on the important infrastructure like databases, backend services and even the forum. Having had the complicated infrastructure we did I simply do not know if every single piece had a firewall in front. This does not mean that this attack vector was used.

1 Like
#14

How are you doing this if you haven’t got a security professional onboard until mid term?
You guys are great at a lot of things, but security is clearly not one.

If you take this serious you will hire someone that knows how to handle beaches.
As I’m working in the business as a incident responder it sounds like you haven’t been hit by a sophisticated state actor, rather a script kiddie.
And those would be easy to detect.

What do you mean by firewalls?
WAFs?

3 Likes
#15

The need to find a new architecture is not uncommon. Monzo has always been very transparent about the iteration of its platforms and published a blog post as recent as March 2022 about securing microservices.

3 Likes
#16

Our engineers are more than capable to secure a simple setup. We just didn’t put enough attention to it as we had a lot of other tasks and the system simple naturally scaled to become pretty big.

We don’t know a lot yet but we are pretty sure that it was a sophisticated actor. It’s not like our doors were simply open just awaiting the first person to knock.

I will not go into more details before our technical team gets a chance to post an update.

4 Likes
#17

Are the issuing of cards on hold due to the recent event? as It has been months since I signed up for the app, I still have not received the cards, I have sent a PM with my details if you could please have a look.

Also in regards to the Amex support, would you know when it would finally be available?

3 Likes
#18

@Koray
Thank you for instantly taking the time to write an in-depth post and promising that you will keep us updated daily till this is over. Very happy this is happening now :slight_smile:
See open communication always helps. I think you will become stronger in the long run from this if you stick to this now. Did you look into pen tests in order to show that you care about security? Remembear dose that quite well for example. (Via cure53)

2 Likes
#19

Yes, we actually do think about pen tests. We will most likely go for PCI compliance in the near future, which includes pen-testing and important other security measures.

2 Likes
#20

I’ll have a look.

1 Like