Site blog title mreza

PROTECT THE DATA!

14/04/2017
Helpprotectcompanydata large

If you were to ask yourself – what is the essence of any software system – what would you say? (that is, what would you have said without having read the title of this post 😃).

There are several things I can think of, such as clean code, DRY methods, integration testing, the technology being used, great design… Still if I had to pick between losing all of those, or just the data and almost my answer would be: “Protect the data”.

Application code and design is generated by a limited number of people and we can be (fairly) certain of who they are. On the other hand, application data is generated by the application’s users. The number of application users tends to be much bigger (fingers crossed). Not to mention that data accumulates over time. For any successful application – data (if or when lost) can not be recreated.

Still, Rails is notorious for its bad relationship with the RDBMS system. Not in the sense that they don’t support it. Rather, in the sense that the ecosystem views things more in a “code comes first” way with data storage being only secondary. This can be read between the lines in trade-offs like the following:

  1. You don’t need first class referential constraint support, Rails does it for you
  2. Let’s add first class support for cascade delete (so you as the developer can test out your code and enjoy the rest of your day)
  3. Let’s add a brand new abstraction on top of the existing SQL which we’ll use to fetch data
  4. Raw SQL is shameful
  5. We should not use RDBMS to its full capacity, but rather we’ll keep our “code” portable with this abstraction on top of SQL and we’ll use only a common subset

Examples From the Wild

One medium sized insurance company managed to forget the “where” part of a fairly simple SQL update script. They ended up resetting ALL invoice amounts to 1 EUR. Thanks to regular backups and transactions logging, they where able to restore their backup and roll back their transactions right up to 1 minute prior to this accidental update. Had they not had both of those, I can’t even begin to imagine what the consequences would have been. Just to clarify things, “Transaction Log” logging is disabled by default in SQL Server and you need to actually know what you are doing and turn it on before disaster strikes.

If you are thinking to yourself, well of course they had backup – they are an insurance company after all – think again.

I also happen to know a case of a decent sized bank which managed not to create backups of their production data for a whole month. How is that even possible? The person who was supposed to put in the magnetic tape was on a vacation. They ended up “creating” their backups with no tape in the backup machine.

Now you must be thinking – wait, this could never happen in a normal country, you guys over there are crazy. But didn’t Amazon just recently have a huge S3 outage in the US-east region – at the start of March/2017 – due to Amazon employees messing things up?

Amazon announced that the cause of the outage was some inaccurately entered code during a debugging, which caused a large number of servers to be removed.

OK, not a database issue, but still data. My point being – these things do happen. Quite frequently.

Rails Active Admin

There’s this “nice” and useful option in the Active-Admin gem (it’s the most popular Rails Administration gem with > 1.5M downloads) to bulk delete all entries.

screen shot 2017-04-14 at 22 42 32

If we were to call this action on wrong items some very bad things would happen. In some cases (Rails) we even go out of our way to make sure the “delete” action is clean. So we say, “Oh, you want to delete user? Let’s delete all data related to this user with this cascade-delete rules. This is nice in theory, but in practice it is a nightmare. Why would we delete the user along with all of his/her related data? What exactly is the business scenario for this? I can only say that while this might make sense for development, it is a big no-no and a nightmare for production data.

Snake Oil Solutions

When faced with reality of how severly uncool it is to delete real world data (which tracks events that have happened in real world usage) the knee jerk reaction is to create a “perfect” solution with minimal changes required.

Welcome to Acts As Paranoid, a gem for “soft” delete. It has been downloaded more than 0.5 million times. While it is a great improvement and miles better than just obliterating data, in a way it just further blurs the issue at hand.

For example maybe we want to set certain users as inactive and remove them from a certain drop down list, but is hiding them from the rest of the application by using default scopes really a good enough approach? I feel this just post-pones coming up with a proper solution – which is supporting “deactivated” items that are still available to the system, but ignored in majority of the day to day functioning.

Leaving Guns Around the House

Personally, I view cascade delete and batch delete as a recipe for data disaster.

That time a drunk Richard Nixon tried to nuke North Korea

The President got on the phone with the Joint Chiefs of Staff and ordered plans for a tactical nuclear strike and recommendations for targets. Henry Kissinger, National Security Advisor for Nixon at the time, also got on the phone to the Joint Chiefs and got them to agree to stand down on that order until Nixon woke up sober the next morning.

We should always aim to protect our system against these kinds of scenarios.

Worst Case Scenarios

Let’s imagine we’ve got ourselves a system that has users for whom we generate invoices. Let’s try to figure out what would happen if we:

a) Do a batch user delete from Active Admin b) Execute “User.destroy_all” in Rails console c) Execute “DELETE from users” in RDBMS console

In an ideal world we would get referential constraints exceptions complaining for those Users who have invoices.

In those case we don’t want to “soft” delete and we certainly don’t want to cascade delete both the users and the invoices!

We need to limit exposure, we need to hide nuclear keys. Database is a fortress!

Remedies

  1. Put in motion a backup/restore plan a) Make sure you have incremental backups dating back far enough in history
  2. Remove cascade delete – both for the database and for Rails
  3. Enforce referential integrity
  4. Support “deactivation” in favor of “soft” delete

It is like digging a trench around a castle to protect valuables inside. Make sure if you want to completely remove data from the database it has to be manual, painful and complicated.

What we want to do is make sure we limit exposure to mistakes that would trigger unwanted deletion and we want our data storage to protect data at the lowest possible level (RDBMS).

There is no silver bullet, no perfect solution. We still have to turn on our brains and hope for the best. Nevertheless we can up our chances by doing things listed above.

Category: Database
OTHER POSTS
OTHER POSTS
Created with Sketch.
Railsios
06/06/2018 / Tin Ilijaš

UPLOAD IMAGE FROM SWIFT 3 IOS APP TO RAILS 5 SERVER

Once upon a time I was given the task to send an image captured on IOS to our Rails back-end server and at first I figured: “Wow this can’t be such a big deal, I know how to take an image and I know how to make a request so why would thi...
Omniauth f1d5b77f6e5fc9adc96fdb4dc3a7d5f3 %281%29
06/06/2018 / Krešimir Bojčić

DEVISE WITH OMNIAUTH FOR SINGLE AND MULTIPLE MODELS – RAILS 5

In this post we'll describe on how to use OmniAuth in combination with Rails and Devise to support authentication of existing and new users without asking for email/password combinations.
Created with Sketch.
Aa 2fa
31/10/2018 / Zvonimir Ivašković

IMPLEMENTING GOOGLE AUTHENTICATOR IN ACTIVE ADMIN

In this blog post we will be showing you how to implement a two-factor authentication solution for Rails' Active Admin by using Google Authenticator in combination with a custom mailer for sending the QR registration code. A couple of o...
Created with Sketch.
Conferencecam group
29/06/2018 / Krešimir Bojčić

LOGITECH GROUP – REMOTE VIDEO CONFERENCING SYSTEM REVIEW

To have a successful remote agency/client relationship you need three main ingredients: 1. Trust 2. Effective communication 3. Delivering results
Global infra 3.30.18.b559f46825615c1ae40f319d0c4d9139fea9c492
29/06/2018 / Krešimir Bojčić

SCALE FOR SPEED AND AVAILABILITY

In this post I'll go over various options for scaling your business web platform. We'll take a look at five different approaches. There is no wrong or right approach, it is just a matter of what aspects you want to emphasize and what you...
Created with Sketch.
Railsandreact
28/06/2018 / Matija Munjaković

SETTING UP REACT & RAILS HAS NEVER BEEN EASIER

Just a couple of days ago Beta 1 of Rails 5.1 got released, bringing with it a slew of new features. Most prominent among them being the inclusion of Webpack. For the uninitiated, Webpack serves as a bundling tool for your project’s Java...
Created with Sketch.
Untitled
03/07/2018 / Krešimir Bojčić

FREE SSL ON AWS OPSWORKS RAILS APP

In this post I'll try and describe how to set up SSL for your Rails app. This solution is free and will automatically extend the certificate once the certificate runs its course.
23 512
06/06/2018 / Tin Ilijaš

HOW TO DEACTIVATE USER – RAILS WITH DEVISE

Sometimes when I find the time (which happens roughly 23 to 37 times each and every day) I visit Stack Overflow, hopefully to solve a problem – but regularly leave irritated and heartbroken because the solution is too geeky and doesn’t e...
Created with Sketch.
7e6dcc68 1ba3 11e7 8286 407dd1a1b50f
29/06/2018 / Tin Ilijaš

SWITCH USERS WITH USERSWITCH

Recently we were hired by a startup and quickly assigned a task to build a brand new feature for them. So no problemo here! The feature was fairly extensive which necessitated thorough testing – perhaps a third of the entire application ...
Created with Sketch.
Untitled
03/07/2018 / Krešimir Bojčić

HOW TO BUILD AN MVP

So, you’ve decided to build a new disruptive product that will earn you glory and money? Congratulations… welcome to the club and good luck!
1 ee4irlninahubiurfqv3fq
29/06/2018 / Tin Ilijaš

8 TIPS TO BECOME A DEVELOPER

Almost a year has flown by ever since I started learning programming from scratch. As a result of that, I am writing this blog post to help new and old developers with some tips I learned in this beautiful world of programming.
Created with Sketch.
Untitled
03/07/2018 / Krešimir Bojčić

VISUALIZE DATABASE SCHEMA – POSTGRESQL DB – RAILS 5

When working on bigger systems it helps having your database model visualized. There are many great solutions but a few of them are free. If you fit in the “small enough” not to use [ER/Studio Data Architect](https://www.idera.com/er-...
Created with Sketch. Created with Sketch.
Hmawawbapi
03/07/2018 / Matija Munjaković

HANDLING MOBILE AUTHENTICATION WITH A WEB-BASED API

Quite recently I had the chance of implementing a log-in/authentication system for one of our mobile applications. Resources available for both the Android and iOS eco-systems are fairly exhaustive but comparisons of the pros and cons of...
Created with Sketch.
Railsios
06/06/2018 / Tin Ilijaš

UPLOAD IMAGE FROM SWIFT 3 IOS APP TO RAILS 5 SERVER

Once upon a time I was given the task to send an image captured on IOS to our Rails back-end server and at first I figured: “Wow this can’t be such a big deal, I know how to take an image and I kno...
Created with Sketch.
Omniauth f1d5b77f6e5fc9adc96fdb4dc3a7d5f3 %281%29
06/06/2018 / Krešimir Bojčić

DEVISE WITH OMNIAUTH FOR SINGLE AND MULTIPLE MODELS – RAILS 5

In this post we'll describe on how to use OmniAuth in combination with Rails and Devise to support authentication of existing and new users without asking for email/password combinations.
Created with Sketch.
Aa 2fa
31/10/2018 / Zvonimir Ivašković

IMPLEMENTING GOOGLE AUTHENTICATOR IN ACTIVE ADMIN

In this blog post we will be showing you how to implement a two-factor authentication solution for Rails' Active Admin by using Google Authenticator in combination with a custom mailer for sending ...
Created with Sketch.
Conferencecam group
29/06/2018 / Krešimir Bojčić

LOGITECH GROUP – REMOTE VIDEO CONFERENCING SYSTEM REVIEW

To have a successful remote agency/client relationship you need three main ingredients: 1. Trust 2. Effective communication 3. Delivering results
Created with Sketch.
Global infra 3.30.18.b559f46825615c1ae40f319d0c4d9139fea9c492
29/06/2018 / Krešimir Bojčić

SCALE FOR SPEED AND AVAILABILITY

In this post I'll go over various options for scaling your business web platform. We'll take a look at five different approaches. There is no wrong or right approach, it is just a matter of what as...
Created with Sketch.
Railsandreact
28/06/2018 / Matija Munjaković

SETTING UP REACT & RAILS HAS NEVER BEEN EASIER

Just a couple of days ago Beta 1 of Rails 5.1 got released, bringing with it a slew of new features. Most prominent among them being the inclusion of Webpack. For the uninitiated, Webpack serves as...
Created with Sketch.
Untitled
03/07/2018 / Krešimir Bojčić

FREE SSL ON AWS OPSWORKS RAILS APP

In this post I'll try and describe how to set up SSL for your Rails app. This solution is free and will automatically extend the certificate once the certificate runs its course.
Created with Sketch.
23 512
06/06/2018 / Tin Ilijaš

HOW TO DEACTIVATE USER – RAILS WITH DEVISE

Sometimes when I find the time (which happens roughly 23 to 37 times each and every day) I visit Stack Overflow, hopefully to solve a problem – but regularly leave irritated and heartbroken because...
Created with Sketch.
7e6dcc68 1ba3 11e7 8286 407dd1a1b50f
29/06/2018 / Tin Ilijaš

SWITCH USERS WITH USERSWITCH

Recently we were hired by a startup and quickly assigned a task to build a brand new feature for them. So no problemo here! The feature was fairly extensive which necessitated thorough testing – pe...
Created with Sketch.
Untitled
03/07/2018 / Krešimir Bojčić

HOW TO BUILD AN MVP

So, you’ve decided to build a new disruptive product that will earn you glory and money? Congratulations… welcome to the club and good luck!
Created with Sketch.
1 ee4irlninahubiurfqv3fq
29/06/2018 / Tin Ilijaš

8 TIPS TO BECOME A DEVELOPER

Almost a year has flown by ever since I started learning programming from scratch. As a result of that, I am writing this blog post to help new and old developers with some tips I learned in this b...
Created with Sketch.
Untitled
03/07/2018 / Krešimir Bojčić

VISUALIZE DATABASE SCHEMA – POSTGRESQL DB – RAILS 5

When working on bigger systems it helps having your database model visualized. There are many great solutions but a few of them are free. If you fit in the “small enough” not to use [ER/Studio D...
Created with Sketch. Created with Sketch.
Hmawawbapi
03/07/2018 / Matija Munjaković

HANDLING MOBILE AUTHENTICATION WITH A WEB-BASED API

Quite recently I had the chance of implementing a log-in/authentication system for one of our mobile applications. Resources available for both the Android and iOS eco-systems are fairly exhaustive...
Created with Sketch. Created with Sketch.
Railsios
06/06/2018 / Tin Ilijaš

UPLOAD IMAGE FROM SWIFT 3 IOS APP TO RAILS 5 SERVER

Once upon a time I was given the task to send an image captured on IOS to our Rails back-end server and at first I figured: “Wo...
Created with Sketch. Created with Sketch.
Omniauth f1d5b77f6e5fc9adc96fdb4dc3a7d5f3 %281%29
06/06/2018 / Krešimir Bojčić

DEVISE WITH OMNIAUTH FOR SINGLE AND MULTIPLE MODELS – RAILS 5

In this post we'll describe on how to use OmniAuth in combination with Rails and Devise to support authentication of existing a...
Created with Sketch. Created with Sketch.
Aa 2fa
31/10/2018 / Zvonimir Ivašković

IMPLEMENTING GOOGLE AUTHENTICATOR IN ACTIVE ADMIN

In this blog post we will be showing you how to implement a two-factor authentication solution for Rails' Active Admin by using...
Created with Sketch. Created with Sketch.
Conferencecam group
29/06/2018 / Krešimir Bojčić

LOGITECH GROUP – REMOTE VIDEO CONFERENCING SYSTEM REVIEW

To have a successful remote agency/client relationship you need three main ingredients: 1. Trust 2. Effective communication...
Created with Sketch. Created with Sketch.
Global infra 3.30.18.b559f46825615c1ae40f319d0c4d9139fea9c492
29/06/2018 / Krešimir Bojčić

SCALE FOR SPEED AND AVAILABILITY

In this post I'll go over various options for scaling your business web platform. We'll take a look at five different approache...
Created with Sketch. Created with Sketch.
Railsandreact
28/06/2018 / Matija Munjaković

SETTING UP REACT & RAILS HAS NEVER BEEN EASIER

Just a couple of days ago Beta 1 of Rails 5.1 got released, bringing with it a slew of new features. Most prominent among them ...
Created with Sketch. Created with Sketch.
Untitled
03/07/2018 / Krešimir Bojčić

FREE SSL ON AWS OPSWORKS RAILS APP

In this post I'll try and describe how to set up SSL for your Rails app. This solution is free and will automatically extend th...
Created with Sketch. Created with Sketch.
23 512
06/06/2018 / Tin Ilijaš

HOW TO DEACTIVATE USER – RAILS WITH DEVISE

Sometimes when I find the time (which happens roughly 23 to 37 times each and every day) I visit Stack Overflow, hopefully to s...
Created with Sketch. Created with Sketch.
7e6dcc68 1ba3 11e7 8286 407dd1a1b50f
29/06/2018 / Tin Ilijaš

SWITCH USERS WITH USERSWITCH

Recently we were hired by a startup and quickly assigned a task to build a brand new feature for them. So no problemo here! The...
Created with Sketch. Created with Sketch.
Untitled
03/07/2018 / Krešimir Bojčić

HOW TO BUILD AN MVP

So, you’ve decided to build a new disruptive product that will earn you glory and money? Congratulations… welcome to the clu...
Created with Sketch. Created with Sketch.
1 ee4irlninahubiurfqv3fq
29/06/2018 / Tin Ilijaš

8 TIPS TO BECOME A DEVELOPER

Almost a year has flown by ever since I started learning programming from scratch. As a result of that, I am writing this blog ...
Created with Sketch. Created with Sketch.
Untitled
03/07/2018 / Krešimir Bojčić

VISUALIZE DATABASE SCHEMA – POSTGRESQL DB – RAILS 5

When working on bigger systems it helps having your database model visualized. There are many great solutions but a few of them...
Created with Sketch. Created with Sketch.
Hmawawbapi
03/07/2018 / Matija Munjaković

HANDLING MOBILE AUTHENTICATION WITH A WEB-BASED API

Quite recently I had the chance of implementing a log-in/authentication system for one of our mobile applications. Resources av...