How To (Properly) Split Up A Large List
Ever need to split up your large list of contacts into random, equally sized segments? Maybe you want to run a split test. Or perhaps you need to perform some batch operations and want to schedule them in smaller segments to run at night over the next few days. One of the most common uses for this is when you're dealing with large groups of contacts that you need to put through a campaign that uses our Helpers, which, in turn, use the Infusionsoft API.
So what's the big deal with large groups of contacts and the API?
When you put a large set of contacts into a campaign that triggers an API call to Infusionsoft what happens is that you may trigger API Throttling. In fact, if you have more than 10k contacts in there, I can almost guarantee that you will. Throttling means that Infusionsoft will see too many calls coming in too quickly and begin to limit how many and how fast it responds. This can trigger delayed responses and 500 errors as well.
Performing an Nth Split…
If you have more than 10k contacts to drop into a campaign that fires an http post to all contacts, then I recommend splitting up your contacts into nth segments. An Nth split means every 3rd record, or every 4th record, etc… It's called an “nth” split because “n” represents the number of splits that you decide.So, for example, if you wanted three equal segments, you would repeat the numbers 1, 2, 3 in the Split column – pasting them down until you hit the end of the file (see the image at left). Notice how the same 1, 2, 3 patter repeats from top to bottom in the Split column.
Using Excel To Create the CSV
I know I said Excel, but you can use any spreadsheet to do this, including Google Sheets, Excel for PC, Excel for Mac or even Numbers on the Mac. In fact, if you could even use a simple text editor like TextEdit or Notepad, or even, heaven forbid, “vi” (if you don't know what “vi” is, trust me, that's a GOOD thing as it means you're not an old unix programmer like me).
The easiest way to do this is to export all of your contacts, but only export the contact id column (nothing more). Then, create a new custom field named “Split” of type “whole number”.
Next, in Excel, fill that second column of the CSV with a series of numbers all the way down for the number of “splits” you want to make. So if you're splitting into 4 segments, use 1,2,3,4 then repeat… 1,2,3,4 etc… down the column. That will randomly (based off the contact id) assign your list into 4 nearly equally divided groups. In this example, I did a 3-way split.
Updating the Data in Infusionsoft
Next, use the Modify Existing Contacts method of the Data Cleanup option in the Admin menu to REPLACE the data in Infusionsoft. This allows you to bypass that whole update and merge contacts issue you have when doing an import. Let me be clear, you are NOT importing the contacts, but rather replacing the content.
Here's what the screen looks like:
Because the uploaded CSV will only contain two columns – the Contact ID and the newly created Split field – you're not really over-writing any data, you're simply appending the values into the newly created Split custom field and locating the right record via the Contact ID.
This is the fastest, least painful, least error prone, most random method of splitting up your list. It's very random because it splits using the nth record approach which means you'll avoid issues like having one segment of “new” contacts and one of “old” contacts, etc… which can yield unpredictable results.
When To Use Our Split It Basic Helper (and when NOT to use it)
So you might be wondering why we're telling you about this method when we have a method for splitting up contacts randomly already as one of our Helpers – the Split It Basic Helper. This is a very useful Helper, but every tool has its intended purpose.
The Split It Basic Helper is designed for simple A/B splitting. It doesn't allow for multiple segments (though that feature has been requested and likely will be added in the future). More importantly, however, is that if you have a large list, greater than 10k contacts, you'll not want to use it for the same reasons discussed above – Infusionsoft API Throttling.
Go Forth, Divide and Conquer!
Okay, you should now be master splitters – so go forth, divide and conquer all of your large batch processing operations in confidence and without slowing your system to a crawl. Remember to use a delay timer and perform large batch operations after midnight so that they don't impact your system performance!
Enjoy… and remember… to get more great tips like this, subscribe below!
