Cleaning up SPA for GDPR

20 May 2018 / Technology / SPA conference

I think GDPR is great. I am really pleased that companies are now being more thoughtful about how they handle my data. For SPA, a conference that has been running for over 20 years, this coincided with a project I had already started to clean up the user data, and here’s what I did.

We had a lot of dormant users

When I first decided to clean up the user data last summer, there were 1176 registered users, of whom 906 hadn’t logged in in the past 5 years. Some of them hadn’t logged in since the account was migrated from a previous database in 2004.

The data we store about users is mainly the information they enter in their profile: name, email address, and maybe bio, company name, Twitter handle etc. We used to also store address, phone number and fax (!) details if the user entered them, but I removed these fields and all the data as part of retiring the wiki.

We also store proposals users have made, and any feedback or reviews of other proposals, but only for the duration of one conference cycle (i.e. they are deleted every year).

First step: delete a large number of inactive users

As Mags Allen points out in this excellent developers’ guide to GDPR, one of the key points of GDPR is only to store the data you need. So the first step was easy; I deleted everyone who hadn’t logged in since 2010. I left a long gap as, because of the community nature of SPA, people might not submit for a few years and then come back into the fold.

Although it wasn’t completely straightforward; the dates were stored as strings and the format had been changed at some point from d-M-Y to j F Y (i.e. from 20-Nov-2004 to 20 November 2004) which made identifying the earlier records to delete somewhat complicated. (Something like the below helped).

SELECT * FROM users where str_to_date( reg_date, '%d-%M-%Y' ) > '20107-01-01'

I changed the format back, so this will even itself out over time as users either log in or are deleted for being inactive.

This left me with 584 users.

Working out what we need consent for

We generally only email our users a few times a year, announcing the CFP, asking them if they’d like to get involved in giving feedback or reviewing sessions, and then to tell them about the conference and suggest they come.

The first two are arguably “legitimate interest” because we have a “relevant and appropriate relationship” to the people on the list, but the last one is definitely marketing. In any case, I think it’s best to be explicit.

So in November last year, I added a checkbox to ask for consent to receiving emails. I highlighted the box so that users who hadn’t seen it would have their attention drawn to it.

I also made it easier for new users to register. Previously when signing up for an account, you were asked to fill in a whole load of stuff including a bio. In fact none of this info was needed unless you were a session leader, and it was offputting and a hassle to be asked for it all up front.

I changed this so on first registering you are now only asked for name, email and consent to contact. If you come to edit your profile later you are given the option then to enter the rest of the info. In most cases, you would only edit your profile if your session is accepted to the conference, at which point details such as bio become relevant.

You could see all the registered users

Because SPA is mostly interactive workshops, many sessions have more than one session leader. One person submits, and then adds the people they will be co-leading with.

If they were new to SPA, you added their email address and an account was created for them. But if they weren’t new to SPA, you could select their name from a drop-down list of all registered users.

Like many aspects of the conference, this dates back from a couple of decades ago when it was mostly a group of people who already knew each other. But fast-forward to the current situation with over 1000 users and being able to enumerate all users is really not great security. It’s also not a great user experience.

I changed this so that you just add the co-leader’s name, and behind the scenes we figure out if they are an existing user or not.

I tested the current list

Once I’d made those changes, in early December last year, I emailed all the remaining users to announce our call for papers, including an unsubscribe link and instructions on how to delete their account. This was the first time we’d emailed the list for a while.

It went to 584 people, had 95 hard bounces (including an entire company!) and 5 unsubscribes which helped clear up the list. (This, among other publicity, resulted in some excellent submissions for the conference this year and it’s a great programme – you should come!)

Four months later, I reviewed who had consented

By April this year, there were 553 users.

77 had said yes to being contacted, 15 had said no, and 461 had neither said yes or no (i.e. they hadn’t edited their profile).

Of the 77 who had said yes, 67 had registered after the addition of the consent checkbox – so would have seen it on the page they created the account.

Of the 15 who said no, 14 had also registed after the change.

It’s not possible for you to say neither yes nor no on first registration, because not checking it saves it as no. However, it is possible then to log into your account and not see it, because it is on an ‘edit my profile’ page which you may not visit if you’ve logged in for another reason.

In fact, of the 461 who still had a NULL value for the checkbox, 23 had logged in since the change but clearly hadn’t edited their profile. Not a huge number but worth making it more prominent, so I also flagged it up on the first page you see when you log in. Over the next couple of weeks, a few more people changed the setting for that.

It was also nice to note that the majority of new users (88%) had consented to email, which was encouraging about the messaging.

Last chance to sign up

Alongside this I’d been updating our terms and conditions with the help of the other committee members.

On 11th May I sent an email announcing the programme. I sent two versions: one to all those who’d checked yes to emails, and a different one to the remaining people who had selected neither yes nor no, which in addition to the details of the programme also said that it was the last email they’d receive from us unless they opted in.

This had a good result; within 24 hours the number of people consenting to contact had gone up to 92, and a week later it was up to 100.

And we’re done for now

There’s some extra stuff I’d like to do, like make it easier for users to delete their own accounts rather than having to ask one of us to do it, but I’m really happy to have cleaned up the user data so we are only storing the minimum information about people, and are only contacting those who are actually engaged with the conference.

And it’s a great conference, you should come!