Programmatic SEO has started to kick off a bit, with more and more people jumping on the bandwagon.
This is leading to more and more tools being created specifically for programmatic SEO.
So what are the tops tools at the moment? What should you use for your new project?
What I look for in a tool
Analysing the top tools, one of the main things I’ll be looking through is what more advanced features the tools offer, verse just using a Google Sheet to hot-swap in different dynamic variables.
Each to their own for this, and there are many that may disagree, but a good programmatic SEO tool with allow some sort of rules that go above and beyond a basic substitution formula in Google Sheets.
Yes, mass page creators have, and still will, make many people good money.
But, a true programmatic SEO tool should allow you to go above and beyond. Offering some sort of rule system that takes the data points into account and outputs different text based on their values.
I’ll also dig into what options are available for data upload & formatting,
PageFactory
Launched in 2022 by Allison Seboldt, PageFactory is a programmatic SEO-specific tool that allows for bulk post creation from your data set.
Let’s dive in and take a look at PageFactory, and see whether this is the pSEO tool for you.
Process
The overall process is super simple to run through. Just a few settings, your data upload, and then your content template and you’re ready to export.
Datasets
You’ll upload a CSV file of all the data points you’d like included, and will then need to manually enter all the column titles of the data set so that they’re usable.
Obviously can get annoying after a while for larger data sets, with an auto-column extraction & assignment being an easy win here.
Content Templating
The PageFactory template system will output one piece of content per row of data. You’ll write out the content template you’d like to use, and then just insert the variables with a little selector from a list you set up in the settings.
The templates run on a per-row basis, meaning a content piece gets generated for each row.
Integrations
PageFactory currently integrates with both WordPress and Webflow, so it’s covering a couple of the top blogging platforms people are using at the moment. It does appear she’s working on a Shopify integration at the moment, however, may be facing some logistical issues with how Shopify integrations work, so you’ll have to stay tuned to find out about that one.
There is also an alternative of downloading the content, rather than leveraging an integration. This would allow you to bulk import the content into whatever CMS it is you’re using.
Summary
Overall, PageFactory is great for a person just jumping into programmatic SEO who wants to play around with creating some content.
This would be perfect for the smaller pSEO projects, or projects where every row has every data point available.
It’s missing some more advanced rules for content creation, but Alison is still working on the tool so no doubt more advanced options will be added in the future.
Typemat
Typemat offers a basic programmatic page creator, where you paste in your data set and then create templates from that.
Process
Typemat gives an easy process of adding data, creating templates, and then posting.
Datasets
You just select all your data, and then literally paste it into the Typemat interface, rather than upload anything.
This is a nice quick option for importing data, but certainly wouldn’t see this working well with larger datasets though.
Content Templating
The templating system is a bit basic here, with a no-rule style addition of the data points.
Integrations
Typemat will only work with WordPress.
Whilst the majority of builds will work with this anyway, it probably won’t be the solution if you’re looking at building with anything else.
Summary
The Typemat videos recommend you to set up your content sections in Google Sheets, possibly with AI if you want, and then upload that data. So it’s more of a CMS to handle the bulk posting, and the piecing together of the content, rather than a ‘raw data to final product’ solution.
Easy to kick a project off, but you may find alternatives more preferred that let you have a bit more control from within the software.
SEOmatic
SEOmatic was launched in October 2022 by Minh Pham and offers both their own CMS you can connect a custom domain too (great option!) or integrations with many of the top CMSs.
Is SEOmatic going to be the programmatic SEO tool of choice for you? Let’s take a look!
Process
The page creation process is a little more in-depth, and may take a little bit longer as there are a few more settings than the other tools.
Considering the integration options that are available though, its a pretty simple process to get some pages live via the tool. Both directly, and via an integration.
Datasets
Importing your CSV dataset will auto-import all the columns, and then allow you to select whether you’re adding it to the data or whether you’ll ignore the data point.
Content Templating
You can create both an excerpt and full-page content, allowing you to specify the summary separately.
There are some advanced IF and IF/ELSE rules available. I’ll have to dig in a bit further to try these out, so will update once I have had the chance to do so.
Spintax is accepted, however, it’s using square brackets instead of the normal ones, {}, that are used with spintax. So you’ll need to reformat your text if you’ve got it in proper spintax already
SEOmatic is also currently testing AI prompting from within the content templating. This will let you use your variables for the prompts, so once you’ve nailed the prompts you can scale generation from within the tool! Great feature.
Integrations
Many. They basically offer integration with anything you’re likely to use, with the current selection including; WordPress, Webflow, Shopify, Notion, Bubble, Prestashop, Framer, Wix, Typedream, Super.so, Popsy.co, Feather.so.
The integration options feature auto-publishing to the platforms, so basically it’s just – create content, publish pages.
Summary
If you’re looking for a few more advanced programmatic SEO page creation features, SEOmatic may be the tool you’re after.
Google Sheets
Google Sheets will always be a go-to for me, as I completely control the input, transformation, and output.
This will let you do the standard dynamic replacement in a dataset by just typing out the content template, inserting your variables where required, and then copying the outputs wherever you need them.
The best programmatic SEO tool
There are a few options, but the best tool is going to come down to what will work best for you.
Do you just need a few dynamic variables switched out, and then the bulk content exported?
Do you need something with more options?
Do you want auto-posting to WordPresss or a similar CMS?
Stay tuned, as this list is still a work in progress. I’ll also be getting more hands-on with some of the tools and trying to build some sample sites out and see what we can build with them.
Subdomain to Subfolder: The Simple Cloudflare Reverse Proxy
For years, there has been a subdomain verse subfolder debate.
One side says this, another side says that.
People have run their own tests, shown that moving a subdomain to a subfolder can improve a blogs ranking, yet so many have been in denial because Google has said the opposite.
Well, Aleyda Solis came out with direct test results, and a lot more people have finally jumped on the subfolder bandwagon.
Most of the time though, running a blog off a subfolder instead of a subdomain isn’t technically feasible.
It’s a pain to set up.
It was a shocking process the last time I tried to do it with a larger site 4/5 years ago.
However, lucky for us things have changed over the last couple of years.
If you use Cloudflare, you can now how you can have a blog installed on a subdomain, yet force it to load, and make it looks like it exists, under a subfolder.
Aleyda’s test
Kicking it off here, this is the tweet that stirred things up again.
The same blog previously in subdomain vs now in subdirectory comparison in SEMrush 👇 pic.twitter.com/SIjh4tWQJx
Clearly highlighting the tech challenges that she’s gone through to get it going, Aleyda shows there’s a definitive growth following the migration to a subfolder.
An almost instant improvement.
There are plenty of caveats that could exist here, as it’s SEO after all…. but once the migration was complete you can see a nice upwards trend.
In some projects like this, I have seen what I dub a ‘new shiny’ effect. New URLs getting a little boost when discovered/migrated too, and then sometimes a bit of a drop off afterwards.
I reached out for an update to see if there was such a dip, and Aleyda was kind enough to provide a new graph;
Before we get into the different methods of setting this up, you need to understand reverse proxies.
Well, the super basics of them anyway…. which is where my knowledge of them stops anyway lol.
A reverse proxy allows you to essentially mask your website’s true file location, and load it somewhere else.
CloudFlare, in general, acts as a reverse proxy by being a CDN. It masks your server’s true location, by forwarding the requests from a Cloudflare ‘middle-man’ to it.
But this runs as URL in, URL out, where the URL is the same before and after the request and only the IP of the content is modified.
We can tweak and override this a bit, so that the URL of the request, differs from the URL of the content location.
So you could have a blog installed on the subdomain, and leave it there, but make it act and look like it’s actually installed on a subfolder.
A request will come through to https://domain.com/blog/ and then the reverse proxy will grab the content available at https://blog.domain.com, and then load it as if it actually exists at https://domain.com/blog/.
It allows you to bypass the biggest tech issue of running a blog on a subfolder, which is managing multiple different technologies installed in the same location.
Well, something along those lines anyway.
Subdomain to Subfolder with Nginx
Two previous blog moves from subdomain to subfolder that I have helped with, involved nginx reverse proxying.
Nginx is a server technology that does server things, but one thing it does is route traffic. You can give it filters or rules, and tell it to send specific requests one way, or another.
It’s like a little middleman that can move your site’s traffic around.
Using this, you tell it to reverse proxy your subfolder requests, to your subdomain, and have it look like everything loads under the subfolder.
Nginx has mostly been a more enterprise-level setup, so there’s a good chance you might not be using it.
There were a few others floating around, but this was the best one I could find to match the specific requirement of reverse proxying content from a subdomain to a subfolder leveraging Cloudflare.
However, it, unfortunately, didn’t work for me. It was close, but the WordPress side of things kept failing.
It took a few goes to work out whether it was the server (Siteground has some fun caching :/) or whether it was the Cloudflare setup.
Modifying the DB directly, some WordPress config scripts, and many other changes, but every time something else would break.
Missing CSS files, bad redirects, and constant server errors. Every time I patched something, something else would break.
I gave up, and called in some dev support.
A dev named Dat came through, and sorted me out.
Steps to setup the reverse proxy
The following instructions will help you get your reverse proxy setup in both Cloudflare, and your WordPress setup.
1. Create the Cloudflare workers
Log into your Cloudflare account, but don’t load up any of the sites, and you’ll see the ‘Workers & Pages’ setting option;
Jump in here, and click ‘create application’
Then find ‘create worker’
Name the first one;
sitename-reverse-proxy
Modifying the sitename to be your actual site name. This name can be anything, but including the sitename can help incase you want to do this multiple times, as each worker could be loaded under any domain.
Click deploy, and it will load in a default script, which we will replace.
Click on ‘edit code’;
Delete the default code, and then paste in the first set of code from below, for the ‘reverse proxy worker’.
Modify any mention of blog.domain.com or domain.com/blog to be the settings you require.
Be careful not to modify any existing, or add any new, trailing slashes or https mentions as they will break everything.
Click on ‘Save and deploy’ in the top right corner, and then ‘Save and deploy’ again on the little popup modal
Repeat the above steps for the redirect worker, named “sitename-redirect-worker”, and get that one deployed too.
2. Setup the Cloudflare routes
Open the website you wish to add the routes for, and then find ‘Workers Routes’
Click on ‘add route’
Create routes for the following two URLs (modifying them to match what you need), by selecting the redirect service worker you created, and ‘production’ environment
*blog.domain.com *blog.example.com/*
Create routes for the following two URLs (modifying them to match what you need), by selecting the proxy service worker you created, and ‘production’ environment
*domain.com/blog *domain.com/blog*
After both sets of 2 routes have been created, you will see something similar to this on your Workers Routes page;
3. Modify the WordPress Site URL
The easiest step of them all.
Load up the WordPress admin area, and jump into Settings > General.
Modify the Site Address, and not the WordPress address, as per the below settings;
4. Add a trailing slash redirect for the sub-folder
After all this, we couldn’t get a final issue solved in the end, unfortunately. The blog homepage was available with both the trialing slash, and no trailing slash. Just the homepage. Everything else works beautfulllllly.
To get this patched up, we load up Cloudflare and head to the Rules > Redirect Rules
Click ‘Create rule’ under the single redirects section
Create a rule that uses the non-trailing slash blog URL version as the incoming requests rule, and the same URL but with a trailing slash as the URL to redirect these requests to
5. Implement a redirect strategy
If this is an existing build you’re modifying, make sure you implement a full 301 redirect strategy! It should just be a simple 301 rule that forwards from the sub-domain to a sub-folder, but triple check it all.
There’s no point moving to a sub-folder if you break everything along the way.
The scripts
Reverse proxy worker
addEventListener('fetch', event => {
// Skip redirects for WordPress preview posts
if (event.request.url.includes('&preview=true')) { return; }
event.respondWith(handleRequest(event.request))
})
class AttributeRewriter {
constructor(rewriteParams) {
this.attributeName = rewriteParams.attributeName
this.old_url = rewriteParams.old_url
this.new_url = rewriteParams.new_url
}
element(element) {
const attribute = element.getAttribute(this.attributeName)
if (attribute && attribute.startsWith(this.old_url)) {
element.setAttribute(
this.attributeName,
attribute.replace(this.old_url, this.new_url),
)
}
}
}
const rules = [
{
from: 'domain.com/blog',
to: 'blog.domain.com'
},
// more rules here
]
const handleRequest = async req => {
// Redirect WordPress login to the subdomain
let baseUrl = req.url;
if (baseUrl.includes('domain.com/blog/wp-login.php')) {
return new Response('', { status: 302, headers: { 'Location': baseUrl.replace('domain.com/blog', 'blog.domain.com') } });
}
const url = new URL(req.url);
let fullurl = url.host + url.pathname;
var newurl = req.url;
var active_rule = { from: '', to: '' }
rules.map(rule => {
if (fullurl.startsWith(rule.from)) {
let url = req.url;
newurl = url.replace(rule.from, rule.to);
active_rule = rule;
console.log(rule);
}
})
const newRequest = new Request(newurl, new Request(req));
const res = await fetch(newRequest);
const rewriter = new HTMLRewriter()
.on('a', new AttributeRewriter({ attributeName: 'href', old_url: active_rule.from, new_url: active_rule.to }))
.on('img', new AttributeRewriter({ attributeName: 'src', old_url: active_rule.from, new_url: active_rule.to }))
.on('link', new AttributeRewriter({ attributeName: 'href', old_url: active_rule.from, new_url: active_rule.to }))
.on('script', new AttributeRewriter({ attributeName: 'src', old_url: active_rule.from, new_url: active_rule.to }))
// .on('*', new AttributeRewriter({ attributeName: 'anytext', old_url: active_rule.from, new_url: active_rule.to }))
if (newurl.indexOf('.js') !== -1 || newurl.indexOf('.xml') !== -1) {
return res;
} else {
return rewriter.transform(res);
}
}
Following the steps above, your blog should now be publicly accessible under the sub-folder, and the admin panel will be accessible under the original sub-domain.
You should be able to directly load the new subfolder, and it’ll work as if you’re accessing the subdomain where it is actually installed.
I’d love to hear how it goes if you went ahead with the change! Both whether the instruction above completely worked for you, and how performance was if it was an existing build you modified.
Just Another WordPress Programmatic SEO Build – Part 5
I’m not counting this copypasta as code. If it was, then HTML links are code too yeah?
Updated the data to some sample data and some links to see what it’d look like, and threw in a centring style, and it isn’t too bad at all.
Since there will be quite a few links though, there’s a bit too much padding.
Fixed.
Now we need to work out how to actually get this going in the Google sheet.
We have a comma separate list of links that we currently use in the page, but how can we turn this into a table.
We need some way to split that list up into groups of 3 for a 3-column table.
I got stuck here for a bit, but then did some Googling and came across this magical formula.
We can just replace every 3rd comma with the row code (<tr>), and then every other comma with the table column code (<td>).
After more tweaks than I care to admit, I pieced together the below;
We start by including the table opening code, then the comma separated that breaks up with every 3rd comma being swapped for the </tr><tr> portion of the table code.
Then we substitute the remaining commas with the <td style> code.
Then throw the table closing code on the end, and we’ve got ourselves a nice link widget.
Well, nice is debatable.
For those that might want it, the formula for a table from a comma-separated link list is;
*ignore my styling, and formatting, and well….. just anything I do that has code involved.
Throwing it into the site to test we can see it works. Works = nice.
Obviously quite a bit bigger than the comma list, so not 100% sure whether this could be better or not.
However, it gives us another option.
Adding a little extra code could give it a little expander option but that’s beyond what I’d want to do now so I’ll skip that.
Think I will just completely skip including this and just shelve the table code. Will keep it in the spreadsheet and maybe return it later.
If I was going to use it though, I’d be going to the content formula that includes the comma-separated list.
Then using a formula, I’d run a count formula that when the comma count is below say 10, the comma-separated link list could be used. If it was higher than 10 it would pull in the table links cell.
Scraping videos
To build out the content even more, we could embed some Youtube videos.
Ideally, you have your own set of videos to mix up the content.
I don’t.
I’m just going to scrape a heap of youtube videos and embed whatever video currently ranks for the combo keyword.
This used to actually work a treat to help a page rank, and I have seen some benefits recently on a few of my sites, however, this is purely to build out the example content and show you how to leverage it.
To do this, we’re going to come up with a heap of keywords to put into a rank tracker (serprobot). We’ll then export the top10 SERPs, remove everything not in the first place, and then remove any non-youtube links.
Anything left over will be included as a video in the pages.
I’ve loaded up a project in serprobot looking for ‘<combo> youtube video’ keywords.
After a few minutes, it’ll complete the rank check, and I’ll be able to export my SERPs report.
I’ll just insert two rows in the keyword column here so the keyword & URLs line up, and then substitute out “keyword” along with sort the list.
Sorting it by URL, we can then delete anything non-youtube, and then delete 2nd position onwards so we end up with a keyword | youtube list like this;
Strip out the ‘ youtube video’ part from the keyword, and we’re back to the combo name, then throw it back into Google sheets and we’ve got a list of 1,000 youtube URLs with their respective combo.
I’ve added a <youtube> into the content template where I’d like to throw the video in.
We now need to craft the youtube embed code, and get that into a column.
Which yes, will look crazy, but, it’s a standard Youtube embed code. This then has the vlookup inside it that goes off and grabs the combo’s youtube video, and then replaces the ‘watch?v=’ with /embed/ so that the embed works.
This then gets substituted out in the intro content section.
Re-uploading the data and we get…
Fancy videos!
Well, kind of fancy. Whatever Google ranked for the keyword, that’s what gets included.
Some are better than others.
Homepage update
The homepage is still looking a little sad with just a short list of posts.
We need to get this updated with some content.
Possibly links out to the animals with a pretty picture, and a heap of text-centric content to pad it out a bit.
I’d love to set up a bannerbear template for this, and be able to generate a text-over-image thumbnail for each category.
That’s pretty and ideal though, so let’s just go with the functional solution for now.
Also probably don’t need to try and think programmatically about the homepage…. can try something a bit manual.
Bit of text, and some chonky links added in.
Much better.
Well, somewhat better at least anyway.
I LIKE IT ALRIGHT.
Additional content sections
Another big part of a programmatic SEO builds’ content is what is essentially content re-use.
This content re-use primarily occurs on search result/aggregation type pages, where you list out and link to the listings/specific content.
Where these links are included, portions of the content are also included, and that pads out the content on these aggregation pages.
You’ll also find this type of content on sites like G2 & Capterra. Each software has its own page, but then they take a chunk of that text and list it out on the ‘best software for x’ type pages.
Think of this sort of content more like a big block of ignored text. It won’t penalise a page, but it might not add direct value when used alone. It needs to be combined with other elements, and is just used to support those other elements.
This build is a bit different, and not at such a scale. Our re-use chunks might be a bit over-used, when looking at the total percentage of pages with a piece of text.
Oh well though, it’s just a concept build anyway.
Let’s see what we can apply to the build!
Animal-based content paragraph
We’re going to write up a custom dynamic element for each animal, and try to feed in a food variable where we can, but more so focus purely on the animal itself.
Something like the general nutrition habits of the animal in question. Maybe what they normally eat, and what their bodies require.
I’ve leveraged OpenAI to come up with generic garbage about each animal;
I’ve then going to get this included in the combo page’s outputs by adding an <animalGeneric> into the conclusion content template, and then substituting it out.
Essentially just a pure little dupe content section, but mixed in with the rest of the content it shouldn’t be tooooo bad, we’ll see.
Might update it to add a dynamic element to it later, or split it into 2 sections and include them in separate parts of the page content.
Food-based content paragraph
We’re going to do a similar thing to what we did with the animals, and add a section purely about the food.
Probably about their nutrition value, but will really just use whatever OpenAI spits out for us.
Created a list of all the foods, ran them through OpenAI with a little prompt, and received a heap of text;
I should edit it, but nah. Let’s just roll with that. Might get an editor to just tweak it later.
Leveraging these additional sections
I’ll throw both of these new sections in the footer section of the combo pages.
They’ll get included in the content template as a variable, and will then be substituted in the content builder.
It will not include a templated content section, the generic section about the animal, the 4th image, and then a generic section about the food.
Two columns are then added into the content builder sheet.
These do a vlookup to the generic content templates sheet, with one matching the animal and one matching the food, and will return the text element if it exists.
They’re now added to the conclusion content section, with the substitutions in place.
Stripped pagination
Finally stripped out the pagination on the category pages, by editing the raw code of the theme.
This will probably get overridden when the theme updates, so I’ll need to get a child theme going, but I can sort that out later.
I tried to also strip out the post inclusion on archive pages, but the content still shows so I am not sure what’s up there.
Reduced the item count to 1 for archive pages though, which is the lowest it’ll go.
Just a single item is included now, with no pagination below it. Better than before so let’s go with that.
The intent behind this is to de-bloggify the site a bit, and to also ‘silo’ it a bit more by containing the links as much as possible.
We’ve got a decent internal linking setup within the content itself, so that is our core interlinking.
If we need more links, that’s where we will try and get them in.
Might even try fully silo it later too by cleaning up the templated header/sidebar links a bit.
Wrapping it up
We’ve now added some new content to each page, investigated an alternate table linking option, along with creating some shiny new pages.
On top of all this, I discovered a silly mistake in one of my linking codes.
I wasn’t substituting the space in multi-word animals & fruits, in the link codes and imagery names.
This led to 404 pages being crawlable, and a lack of imagery on heaps of pages due to an incorrect lookup.
Rookie mistake.
In the words of a wise man, DOH!
Taking a look at current performance though, we can see a few things.
Crawling is spiking, but a few days delayed so that spike is a week old now.
Google’s started indexing it all, slowly but surely.
16 in mobile usability report. I’ll go into why I look at this in a separate post at some point.
Impressions are starting to kick off now, because….
A heap of keywords are hitting the board. 35 pages with impressions in gsc. Decent little start.
Oh, and I added another 150 pages.
Stay tuned for Part 6, and see what we can do next.
So you’ve got a shiny new programmatic build, and want to know how it’s performing.
Monitoring indexation, traffic, conversions, rankings, there’s so much to look at so what should you focus on?
Let’s take a look into a few things you should be monitoring, and how to go about it.
Key metrics that can gauge performance
The key metrics you should be monitoring will change through the life cycle of your build.
A build that has just been launched, should have more emphasis on tracking indexation, rankings, and even impressions.
For existing builds, or as a build gets older, more emphasis should be placed on two other key metrics. Conversions & clicks.
Everything else is still important, particularly for understanding what modifications could be made to improve performance, but if your setup isn’t driving converting traffic, then what’s the point?
Unless you’re a build that’s purely advertisement reliant, of course.
Indexed Pages
The overall count of pages indexed. This can be monitored via the coverage report, and XML sitemaps.
Ranked Pages
Kind of an extension of indexed pages, ranked pages is the count of pages ranking in the top 100 for the most related keyword possible. You load up a rank tracker with keywords exactly matching specific pages, and then monitor them showing up in the top 100. It’s great for ensuring that Google is actually ranking the pages, albeit low to begin with, rather than just reporting they’re indexed.
Ranking Performance
Most will just use average rank for rank monitoring. Whilst it’s great, it’s unweighted. Keywords with 10 volume will have the same value as keywords with 10,000 volume. I prefer to use ‘estimated traffic‘, which takes into account both the ranking, and the search volume for a keyword, giving a much stronger indicator of ranking performance.
Clicks & Impressions
Pretty obvious here, but clicks & impressions are a clear indicator of performance. Can be broken down by keyword level (albeit filtered data) and URL level, it can give us quite a bit of drilled-in performance data.
Conversions
Last, but certainly not least, conversions. Provided you’re not an ads-driven programmatic build, conversions should be the key of all key metrics. It’s the most direct indicator of the actual performance of the traffic. If it’s good traffic, it’ll convert. Bad traffic, won’t. Even a badly optimised conversion funnel will get traffic into the funnel. These conversions can be broken down into different levels, from funnel entry to funnel exist, ie add to cart verse purchase, to get even better data.
Breaking the metrics down
When analysing the performance of a programmatic build, you can’t look at it at a page level.
You’ll spend hours upon hours looking through the data.
You need to analyse it at the category/site section level. Grouped performance data essentially.
Look at the data for sets of the pages at once.
You could break the data out by category, sub-category, location, or even analyses like ‘location pages verse generic pages’.
Any way you feel it could be analysed that returns a smaller amount of overall groups.
Analyse keyword data by categories
You should group your keyword data into categories and view your performance at the category level. Aggregate all the performance data, and view key metrics like clicks, impressions, and estimated traffic performance by category.
Analyse traffic and performance data by site sections
Just like you group keywords into categories, you can use the same setup to categorise your landing pages into site sections.
Another trick here is to individually verify GSC properties of each site section. If you do that, you’ll get fully granular data for each site section, in its own property. Everything will be filtered to the site section.
Tracking a programmatic build
On top of the standard conversions/clicks, there are a few extra ways I look at performance, especially for a programmatic build.
As long as the GSC is verified before launch, analytics setup with conversions, and you have your rank tracking set up, you’re golden for launch. Everything else can be set up based on the historic data.
There are a few things you can start to look at after a launch, with some of the things I look at being below.
Identify top keyword templates and generate tracking keywords
You might be wanting to track thousands of keywords for the overall project, which is great, but sometimes it’s a bit of data overload.
Focussing on a small subset to gauge overall performance can really help you get a quick analysis done.
I recommend identifying your top keyword templates and then generating a subset of keywords for each main section. I personally track between 100 & 500 keywords per site section/page type.
So for the ‘buy’ channel of real estate, you’d have;
real estate for sale <location> properties for sale <location>
and then ‘rent’ might have;
rental properties <location> properties for rent <location>
Using these templates, pick the top 100 locations, or whatever the variable is, and generate the keywords. You could use merge words, or follow my guide here on bulk keyword generation.
Load these up in a rank tracker, in either different projects, or tagged separately, so that you can quickly view the overall group performance.
So that I can view averages just on the dashboard, and quick view performance for that whole category.
You might also track a subset of generic verse property type keywords. Generic would be;
property for sale <location>
but then property types might be;
houses for sale <location> apartments for sale <location>
And you could do a similar thing with generic vs location keywords.
Just follow the same process as above, and generate the keywords and then load these up in the tracking.
Monitor your indexation
During the initial phases of a build, monitoring indexation is a great way of understanding how Google is initially reacting to it.
Indexing is the first sign of happiness.
An indexed page means it’s passed the initial checks by Google, and they at least, somewhat, found enough value in the page to index it.
If you’ve never worked on a large-scale programmatic build, the amount of pages that don’t get passed the discovered/crawled stages sometimes is incredible. So passing this stage is a great first step.
You can do this through the coverage report at the top level, and by site section if you’ve verified the individual site sections in their own GSC property.
Another good method is creating site-section-based XML sitemaps. These can then be clicked in the sitemap report, and you can view the coverage report based on just the XML sitemap URLs.
The final method of initial is by keeping an eye on the keywords with a URL ranking in the top 100. These should generally be the most related page to the query, so just watching the quantities of keywords with a URL ranking, as a per cent, can give a good indication here too. It might be a bit more performance/quality related than direct indexing, but everyone knows it gets pretty junky after a few pages anyway, so even a shockingly optimised page could pop the top 100!
Count of URLs with impressions
Not so much a key metric, but another great view of performance, is monitoring the overall count of URLs with an impression.
A more ‘advanced’ method of monitoring indexation, keeping an eye on the count of URLs with impressions is a great way to monitor the overall performance growth across the entire build.
Somewhat useful at the start, and particularly great for self-expanding programmatic builds, this metric will help you confirm whether the wider system is driving the performance, or whether it’s a small subset of URLs that are performing the best.
It’s great for ongoing monitoring, especially when you start to break it down by sections and page types.
Analyse the per cent of live pages that are actually generating impressions.
So many metrics, so little time
Keeping what you’re tracking to a minimum, will help you avoid data overload.
Think about what metrics you can’t backdate, like ranking information, and ensure they’re all setup just in case you need them one day.
You never know what you’re gonna need, and when.
Programmatic SEO: Integrating Blog Content With Programmatic
For programmatic SEO, blog posts enable the ability to target specific keyword sets that you just can’t efficiently target in bulk.
They help fill the gap.
How can you best leverage this content though?
How blog content works for most sites
The majority of websites with a blog will throw all their content under /blog/ or just single folders for each content type.
So you might see /advice/, /guides/ and /news/ commonly floating about.
All the blog content sits within these folders, and only in these folders.
You’ll find links to the posts around the site, but all readers once they click will be sent to that static blog folder.
These posts will then link out throughout the site, when it suits, mostly via manual contextual links.
How to improve blog content handling
Having blog posts set up like every other site still holds significant value, and is by no means wrong.
It’s the easiest method of getting gap-filling content live, so you can begin ranking for and driving traffic from additional keywords.
However, you can squeeze a bit more value out of this blog content.
Not by modifying the content itself, but by where the content is used on the website.
By relocating your content to sit within your programmatic directories, you can improve its effectiveness at passing value into those directories.
Rather than just a couple of links from the post into your programmatic setup, you’re shifting the entire value into the setup.
All value your content then generates, is directly passed into the directories where it matters, rather than to your domain, and then back down.
Not only improving the strength within these directories, but also helping aggregate all related content together for Google in the hope of being seen as covering the topic more thoroughly.
How this can be achieved
There are a few ways to integrate this, and you don’t have to give up all those lovely WordPress features either!
Definitely don’t need a full custom blog set up to make this happen/
Headless CMS via WordPress API or similar
WordPress offers an API that allows you to de-couple it from itself.
You can have the admin work as per normal, and then the front end be built however you like it, without all the WordPress bloat!
The developers need to build off the WP API, and can essentially leverage the blog content however they like.
It would work exactly like any other content type you have, so they get full access to it all.
No funky tricks to make things work.
Built it how you want it.
Sometimes devs will request something in the URL to be able to determine the content requested is blog content.
It’s the easiest way to integrate this, but it shouldn’t be necessary.
There are ways to make something like this work, even if it’s maintaining a list of all possible blog posts that are available, and thus then knowing all their slugs and not needing a key anymore.
You could ensure blog posts only get included in these category setups once a week, and then every week you recreate the list of all posts available.
Another possible way is it you know what variable values you’ll have within the programmatic structure, anything left could be deemed to be a blog post, and if not, it could then 404.
You might just need to heavily cache the blog content as it could be a bit of a load on the server on the initial load. But hey, it’s blog content. Not exactly requiring live updates here or anything.
There’s always a way if ideas are bounced around with the team enough.
Reverse proxies with Nginx
This is the only way that I know, that allows for integration between CMS platforms.
Essentially the setup just says that say /buy/ points at one system, and /buy/guides/ points at another.
It requires a unique key in the URL to be able to distinguish the two systems though.
Something like /blog/, /advice/ or /guides/ in the URL of every piece of content, whether it be categories or blog content.
Examples of content integration
Airtasker
Airtasker has now fully integrated their blog content into their structure, by using a
They have their main categories of landing pages like Cleaning, like this;
airtasker.com/cleaning/
But then they’ve managed to pull in their related blog content, under the main /cleaning/ structure like this;
Today, we’re spending the entire part on a single topic.
Imagery.
Like a portion of the last part regarding the AI-generated text, this probably won’t actually apply to you.
You’ll probably have your own imagery to use, or at least, acquire proper images to use.
However, this one is fun. Promise.
Stick with it, and join me as we tinker on the edge of the dark side.
Text-to-Image Generation
Not to be confused with AI Image Generation, which I’ll get into shortly, text to image generation is exactly that.
Generating an image from text.
Yes, I know. Its how AI image generation works.
But this is literally taking the text and putting it in an image.
You build out a template, throw it some text and any images you want included, and an API will spit out a shiny new image.
Normally used for social media banners, programmatic SEO builds have started to leverage this to generate unique imagery related to their content in a less-than-greyhat way.
However, it’s not something I have actually leveraged.
Been on my to-do list for quite a while!
Time to get it going.
Picking a text to image generator
Yes, you can either code this yourself, modify one of the available code snippets out there, or even just buy one! There’s probably a plugin available.
But I’d rather find a good SAAS that does this well for the project.
After doing a bit of research, I’ve decided upon Bannerbear, and their image generation API.
Figured it also suited the whole animal theme.
They’ve got a nice, pre-formatted demo on the homepage that gives you some options.
These options are then spitting out;
A pretty neat way of generating a custom featured image for a post.
The query string method has been updated, to an apparently ‘simpler’ method, but let’s take a look at the whole process.
An alternate, and a bit cheaper, could be using placid. Seems they have some good integration methods too.
Creating a template
Once you’re signed up, you can start by creating a new template. The template will be the shell of what you’re generating, and allow you to hot-swap the different items out.
They offer quite a few templates, so I picked one that suited me and modified it as required.
I’ve included 4 images on the banner, along with the logo and a couple of text elements.
The 4 images will be swapped out, along with the title on the graphic.
Half jumping forward a bit with what the 4 images will be, but you’ll find out in the rest of the post.
As for the title, that will be the combo name.
Generating an image via URL
Following the tutorial here, to change the title and generate an image we get a URL of;
This works perfectly. I wasn’t expecting that at all.
Make sure you follow the tutorial in the post above on how to set it up, as you just need to edit a single bit in the code to tell it where to save images.
After that, just fill in the image name (no extension), the generation URL, along with what’s being replaced.
So for my example above, it’s just going to replace the question text.
Unfortunately, you’re rate-limited at 10 requests per 10 seconds, which yes, you could hit.
We have to modify the Excel script by adding some lines. When you’re in the editor modifying the file name (see in guide linked earlier) you need to look for;
Next i End Sub
Insert 2 lines above that, and add before it;
‘~~> Pause for 2 seconds between requests Application.Wait (Now + TimeValue(“0:00:02”))
The code will look like this;
It will add a 2-second pause before each request.
However, I just realised that Bannerbear requires the darn $150/month plan to do this simple URL generation. You get 10,000 images rather than the 1,000 I was going to play with, but I’m not upgrading for that.
New method time.
Generating an image via Airtable
They offer an Airtable generation set-up too (tutorial here) where you just throw all the fields in an Airtable and then it will build everything for you… proper no code!
You just map the table headings to your field names, insert your data, and then add your Airtable API key to Bannerbear.
Then when setting up the import, you insert your Base ID and table name, and then Bannerbear gets full access.
Here’s my setup before import:
Bannerbear will then take it all, run through them all, and then spit out the image URLs in the final column!
Pretty cool.
I’ll use the Excel image downloader on these URLs now so that I can properly rename the images, but generating them this way was faster and easier anyway!
I also got to keep my question mark.
I’ll sort all of that after I sort the images that will actually go into these, and the pages.
Now the fun bit.
AI Image Generation
Dall-e 2
Have you heard of Dall-e? And in particular, dall-e 2?
Dall-e takes a text prompt, and turns it into an image.
Their homepage demo gave me an option of a prompt of;
Teddy bears working on new Ai research on the moon in the 1980s
Which outputs an image of;
Pretty cool.
Can you see where I’m going with this?
Unfortunately, Dalle 2 is in super limited beta access.
Even if you get access, you can ‘only’ do 50 images a day.
Midjourney
Another ‘top tier’ option is Midjourney.
They’ve recently entered beta, and work via discord where you make your request and it’ll build the image in front of you. You can join the beta here.
When you run a generation, you get an initial set of 4 images.
You can then pick one, and get 4 more, or refine a specific one.
I’m going to try and get 3 new versions of the second image.
Lets now upscale the first one, and see what it spits out.
After it spends a minute generating the image, we get this thing of pure beauty.
It’s definitely more ‘artsy’ than what Dalle-2 is, but it could certainly work in the context of a ‘dogs eating grapes’ page.
Dall-e Mini
There’s an alternative called dall-e mini, which is basically just ‘simpler’ version of Dalle-2.
A very very simple version.
Let’s throw in the same prompt as what dall-e 2 gave me and check the result;
I mean, it’s so close haha.
However, it’s free and easily accessible.
So it is a massive step up from anything else we can have access in bulk to at the moment.
Let’s take a look at an example that directly relates to our topic.
Just for comparison, here’s the exact same output from a dall-e 2 beta account…
You can heavily tweak the prompts though.
There is some great info starting to popup, about throwing different elements on the end.
dog eating a banana cartoon
dog eating a banana sketch
dog eating a banana, studio lighting
I ran through a heap of options, trying to work out what would work best before finding something I liked.
Yeah, it’s not amazing, but it’ll do.
Bulk generating imagery
Now, in case the plan didn’t click, I’m going to attempt to bulk generate images for all the combo pages.
Dall-e Mini will throw out 9 images per request so we’re gonna hook into that.
There are mentions of a dall-e mega trained model, which would be a bit more ‘refined’ than the images we’re seeing above.
However, that takes a bit more work to get going. So for the initial build, we’re going to aim for just grabbing this output and throwing imagery in an S3 bucket.
I’ve got one of the developers I’ve used previously to build something for me that’ll do this. Might look at sharing it later, won’t for now though.
I know I said all in Google Sheets… but… you obviously would never, ever, ever, need to do this sort of thing.
You’d have completely original imagery and would never use AI to generate them.
So it doesn’t count. This is just me filling in the gaps to better show you how to leverage your own content.
Safe to say this is my new favourite thing.
This is essentially a folder full of folders. Each folder is for a specific animal & food combo.
Each of these combo folders, then contains the 9 images that were generated by my prompt.
HOW COOL.
I mean, look at these little fellas just enjoying life chilling with some strawberries.
Each request takes 2-3 minutes, so let’s go with 3 minutes. If we have 1,000 combos to generate imagery for, we’re looking at 50 hours to generate 9 images for each combo.
That’s not too shabby for generating 9,000 somewhat-usable images.
Probably won’t need all 9 for each post, and I certainly don’t want to go through 1,000 folders of imagery to manually select the best.
We need to prioritise the imagery somehow.
Google Cloud Vision AI
Google used to have something where you could put an image up, and it would tell you what it thinks it was.
You can throw the images in there, and it’ll output what it thinks it is;
Both the dog and the banana are clearly recognised.
Unfortunately, however, if we click over to the ‘safe search’ tab, we can see that Google is determining it as ‘very likely’ of being ‘spoof’.
Now I can’t find much quick info as to what this exactly means, but seems to just be their fake image detection.
Google knows it’s faked.
Massive sad face.
But then we look at this outputted image…
This one is marked as ‘likely’ now.
If we generate some others, we can then get it too;
So now it’s only flagged as ‘possible’.
Maybe if we generate enough, it can get even lower.
After testing a heap more, I couldn’t seem to get dogs & bananas any lower.
Something about bananas doesn’t seem to work properly with AI haha.
However, this potato & cat image managed to get this;
So it’s clear you can fool the vision AI detection API with a potato butt cat.
Obviously, there’s a chance Google is using something completely separate in their algo.
There’s also a chance they’re not.
If people swear by the NLP API to optimise text, why can’t we leverage it to choose our imagery?
Image prioritisation with Vision AI
How can we use all this to pick the best x images?
Knowing that we can get at least a portion of images passed the auto-detection, we could filter our imagery and give the images a score based on whether the animal & food are detected.
Another score could be based on the level of spoof the image has come back with.
Mocked up a quick Google sheet template, and will have to also throw this over to the dev to handle.
I’ve then thrown together a quick quality scoring system.
If the animal is detected, the image gets a +2.
If the food is detected, the image gets a +2
Then a score depending on the spoof level.
The scores are just all added up, and I get a sortable score.
Once sorted, there is a ‘sortID’ column which is the following formula;
=IF(D2=D1,F1+1,1)
All it does is count up from 1, when the combo is the same.
Then there’s a formula in useImage which just tells me if I should use the image or not.
=IF(F2<5,“Image”&F2,)
If the sortId is less than 5, then put Image<sortId>. So, Image1.
When it’s time to use these images I know there will be exactly 4 images flagged, and their names will always be Image 1-4, and not the source image number, to avoid confusion.
The dev will just do a mass scan with the cloud API and then throw me back a CSV.
Another thing that I’ll have to rely on a dev for, but another super quick thing to do. You could do it yourself too if you want, but I’d rather just hand it off for a few $.
While that happens though, the image scraper has been working in the background and we’ve got images for 500 pages ready to rock and roll.
Embedding imagery in the posts
Now that we’ve got our images, we need to be able to get them into our post data, and be able to embed them.
I’ve added the image folder to the Image column, which contains ‘Image1’ up to 4.
This way, we can now vlookup from the main datasheet, 4 separate times. Once for each image.
Created 4 different columns on the datasheet.
You’ll see some blank spaces, the images errored out.
So basically, if there’s no image there will be no image link.
Exactly what we need it to do.
Now we need to take these, and build out the HTML code.
I could build the HTML code in the actual content for the page, but that’ll be a bit dirty and won’t help to troubleshoot.
Rather, I’ll add yet another 4 columns. One for each HTML of the image.
This way, I can just slap the cell reference into the content, and do any tweaks separately.
We can’t do this all as one, because the images won’t be a big slab of imagery. Well, that’s not the aim anyway.
We will want to insert the imagery separately, so one at the top, one at the bottom, and then two in the guts.
Let’s start with a formula to check if the link exists, and then will just use the link if it does, and keep the cell blank if it does’nt.
=IF(len(T2)>0,T2,)
We can then modify it with a standard HTML link code, to dynamically insert the image URL from the image1 cell, along with create an alt text that combined the animal and the food.
=IF(len(T2)>0,”<img src=”””&T2&””” alt=”””&proper(D2)&” with “&proper(E2)&”””>”,)
Be sure to use 3 quotations so that they stay in the outputted text. Since sheets is looking at the quotations as the beginning and end of text, it will error out unless you do the 2 for the inclusion of one, and then a third to finalise the text portion and start the formula portion.
Duplicated it across the other 3 columns, and mixed up the alt text a little bit in each of them too.
Now we gotta stick them in the content.
I’m going to leverage the content templates we built out for this one, and just insert <image1> wherever I want to put the first image.
It can go after the first paragraph, but before the question answer.
Image 2 can then go after the first answer.
It didn’t end up quite as clean as I was hoping, but I have inserted the 4 variables into the text templates.
Have then done substitutions on the content sheet to swap out the variable for the HTML from the data sheet.
Probably should have done that HTML code on the content builder to save the vlookup, but oh well, it’s done now.
Uploading the new data with images
I’m not exactly sure whether this will work with the HTML code out the box, so rather than toying around I’m just gonna give it a crack.
Going through the import there’s a step in WP Import/Export that allows for the importing of imagery;
Not exactly sure how this works though, so I am just going to continue the import of the HTML with the S3 references, and maybe we can update so that it actually imports new imagery a bit later.
That’d be cool though.
The upload took a little longer than before…
Almost 4 minutes compared to the normal 30 seconds. Bit more to process now I guess.
But, IT WORKS!
Beautiful.
Except due to the image quality they’re pretty small, and left aligned.
Let’s update the HTML code to centre them, and make them a bit bigger.
I’ll make them 400×400, about double size. The new formula is;
Remember those double quotes to ensure they’re included in the output.
Now to reupload it all and check it.
Looks much better now.
Oh, and bonus!
It actually imported the imagery from the HTML, didn’t have to do anything else. Thought I was going to need to specify the URLs separately or something.
Bloody ripper!
Including the text-to-image featured image
Now that we’ve sorted out images for the pages, we need to sort the featured graphic we built out in banner bear.
I loaded up the 4 images per page into the Airtable we made, then jumped into BannerBear to run the generation
500 images were generated in about 5 minutes.
Well, there goes half my quota!
Completely worth it though.
How about those crazy cat eyes though!
Demons.
Now we gotta get them thrown into our spreadsheet, so I’ll just add yet another tab that can keep track of all of them.
I was thinking that maybe we could just import from the bannerbear URL.
Unfortunately, though, all the images will have a jibber jabber name of random characters. More like an ID than an image name.
I could be lazy and just use that, but let’s do this right and give them a proper page name.
I formatted the titles to turn it into a file name (without extension) so we can use the excel image downloader.
We can throw these names into the downloader with the bannerbear URL.
And then just run it… and wait. It’ll be a while.
The images all come out at like 600KB though, so I need to try and compress and/or resize them.
Free, and you can just upload a tonne of images and then it’ll optimise and you download.
I resized all the images to 800px wide, rather than 1200, which halved their file size.
The tool downloaded them as pngs though, so I then converted them to JPG and it dropped them by 70%, and they’re all pretty much under 100KB now.
That’ll work perfectly.
Not amazing quality, but definitely plenty for what we need at the moment.
Threw them all in S3, and now we just need to reference them in our data.
Knowing the folder where the images is, along with the name, we piece a URL together by adding the folder along with a .jpg extension on the end.
Now just need to vlookup this from the main data sheet and we’re set.
Time to actually use these images somehow.
Figured I’d try put it as the featured image to test on one of the posts.
Works perfectly right out of the box, so we will just go with that.
The WP Import/Export plugin doesn’t have a separate field for the featured image, but there’s a box you can tick that sets the first image as the featured image.
Sounds dangerous if there isn’t a banner bear image available, but seems like it’ll sort it for us provided we enter the featured image URL in the image URLs box.
Continuing with the import we just select it all and wait for the site to download from S3.
Worked like a charm.
Image quality is pretty shotty though so I might try to step the compression back a bit if we can.
The content there is a little more advanced, as it is leveraging rules to determine which content template is used.
I thought I’d take it back to basics, and create a dynamic content generator that was a bit simpler.
A straight-up data-to-content text replacement.
Dynamically generating text in Google Sheets
The data set
The generator starts with your data set.
Each column represents a different piece of data.
You wouldn’t need to use every single of pieces of data you put in the spreadsheet, however, you will need to ensure any data point you do want to use is there.
Each of the columns is a variable name, and that will be taken by the generator and replaced out.
The dynamic text template
On the second sheet, is a cell where you write out your text template.
You just write out the text you’d like to include, in the format you want it, and include the variables wherever you’d like them.
Next to the text template is a list of all your variables.
These are just a list of every header name from the first sheet, and it’s a great list to help you remember what you have to work with, rather than needing to flick back and forth between your data set.
Since it’s formula driven, you can’t copy/paste the variables. However, if you’re not adding more columns to the dataset you could paste the raw data so that you could just copy them in.
The content generation formula
You’ll find the actual formula that does the replacement on the main generator sheet.
The formula might look a little daunting, but it’s just a large nested substitute.
Each heading has “<” and “>” added on either side to convert it to something to use in the text templates.
The formula will then take these variables, and substitute them for the value in that column.
It will then repeat the process.
To add a new variable in;
1. Insert a column before the contentOutput and fill in your data.
2. Add an additional SUBSITUTE( at the front of the list
3. Copy the data after the 2nd to last bracket, and paste it after the last bracket
4. Modify the cell references to instead reference your new column
So if you inserted one into the current template, the formula would go from;
These words give anyone involved in UX nightmares.
SEO first, users second.
SHOCK. AWE.
Oh, the humanity!
But that is exactly how I approach a fresh programmatic build.
That is how I have helped build out multiple successful programmatic builds.
Even though many SEOs will hate this strategy, it is what I will continue recommending for clients tackling a new, large-scale setup.
Why?
It works.
The SEO first, users second, approach
An SEO first approach does exactly that, it puts SEO ahead of the user experience.
It’s not saying to ignore the UX of the site, by no means.
It’s just making sure that anything being built for an MVP (minimum viable product) has SEO front of mind.
If a feature is being considered there’s one simple question that is asked.
“Is this primarily for SEO, or for users?”
If it’s primarily for SEO, great – it’s in!
If it’s primarily for the users, then it’s moved down the task list a little bit.
As time goes on, more focus can be placed on the users, that the system is now generating.
A product could be launched that’s 80% SEO and 20% for users.
2 months later, it’s 70/30. 4-6 months down the line, 50/50.
As the traffic grows, that traffic starts to be optimised for.
Similar to a chicken/egg scenario, except, what’s the point in primarily focussing on users, when they don’t exist yet?
Why I recommend this strategy
Development time is precious.
Programmatic builds take time to get traction in the market.
Getting the MVP live as soon as possible is critical to give the build time to grow in the market.
Getting the MVP live ASAP lets you further iterate on the build once you’ve enabled the ability to drive traffic.
Focussing on SEO first gets you to market faster, with a product that is “good enough” to start getting indexed, and hopefully ranked.
What’s the point in building a pretty page, with fancy animations & conversion-optimised widgets, if you’ve got no one visiting the page?
Avoid the void
A less-than-average user experience also makes sure that the product teams come back to rework the build later.
What’s going to be easier to get across the line.
Reworking a build based on a poor UX for an ever-growing user base?
Or reworking a build based on poor SEO that may or may not actually drive traffic?
I know what one I would rather try to sell into the product team.
You probably won’t even need to mention it. Someone higher up might eventually spot the build and start asking questions forcing more time to be spent on the product. Perfect.
Avoid the void of “that can’t be prioritised” that many SEO projects will fall into.
Get the results, and get more work done, albeit a bit cheekily. Sometimes you’ve just gotta play that game, to get the best result possible.
Keeping existing users out of the SEO-first approach
If you’re an existing website, but planning a new programmatic build, how do you avoid the existing userbase entering the build?
Especially when it’s an SEO-first approach, you don’t normally want to push your existing users into it as it has a higher chance of degrading their experience.
After all, you’re focussing on acquisition more than retention.
The main way of doing this is by keeping internal links to a minimum.
Not something you do with most builds, but at the start just limit where you’re linking in from. Prioritise links from ‘less visible’ locations initially.
Link from within widgets a bit further down the page, in places that users aren’t going to actively look at.
Some systems I have worked on have even only had a handful of homepage links pointing in, from towards the bottom of the page.
It’s enough to kick things off!
This will essentially help you dark launch the site, allowing you to start to gain traction whilst you still build the whole experience out.
No point delaying everything when you can get it live much earlier, with minimal risk.
Should you be taking an SEO-first approach?
Scenario dependant.
(my new fancy way of saying “it depends”)
Are you a brand new site? Yes.
Are you a brand new programmatic build, tacked onto an existing site, that existing users won’t need? Yes.
Are you a brand new programmatic build, tacked onto an existing site, that users will need (ie an optimised search)? No.
Are you rebuilding an existing programmatic build with an existing large user base that will get migrated over? No.
Plenty of great use cases for an SEO-first approach, but it’s certainly not for everyone.
For many, the users should definitely still come first.
Prioritising SEO tasks is key to being able to get work completed by a development team in an efficient manner.
Rather than just going “we need them all done”, or “they’re all a priority”, breaking your tasks into individual priority scores can help them gauge what’s most important.
A lot of guessing is involved, but I like to think of it as educated guessing.
Let’s break down task prioritisation, and how you can better request SEO work with development teams.
The ICE score
If you haven’t heard of it before, ICE scoring gives a simple way of scoring a task 3 ways, to give it a priority.
Impact – How impactful do you think a task will be?
Confidence – How confident are you of that impact?
Ease – How easy a task will be. Essentially the effort, but a reverse score.
Similar to the RICE scoring method, but one less variable!
The ICE score allows you to essentially create a sortable list of tasks.
How to prioritise SEO tasks
You probably have a list of SEO tasks required already, and just need to prioritise them.
Give each task a score out of 10, for each of the 3 variables – Impact, Confidence, and Ease.
Then add the score up, and that is your ICE score, out of 30.
Knowing what score each item variable requires comes down to experience.
Experience in implementing tasks.
Experience in working with developers and knowing what they say to different types of tickets.
The big thing here is, that it’s just a guess!
You don’t need to be accurate.
Since all the scores are based on what you know, you’re essentially weighing up each task against each other task which is essentially what we want.
If you really have no clue about the ease of implementation for a task, then you could sit down with the devs and ask them for a hand.
Let them give you the ease scores themselves after a high-level chat about each task.
It can always be edited later, which will reprioritise the tasks.
Breaking down complex tasks into simple ones
One of the fastest ways I found to get SEO work done was to make the complex, simple.
Take the large, complex tasks, and break them down into a few tasks.
The initial task should be seen as a ‘foot in the door’ task. What is the absolute minimum amount of work required, to achieve the result, with minimal negative results?
Any other “sub-task” related to that task can be seen as upgrades.
There are so many instances where you just need to get a little piece of the overall task complete and live. Then you can either reap some rewards, or make further related tasks more attractive for a development team to pick up by simplifying them.
Lots of “simple” tasks can sometimes achieve more than a single larger task.
Working with development teams
Depending on whether you’re in-house and working with an in-house dev team, or whether you’re agency-side working with an external team, it’s not a straight task list hand-off, unfortunately.
You’ll need to work with the development team, probably a product manager, and walk them through the tasks.
You might need to “sell” them on each individual task, if they’re that picky.
It really comes down to what information you’re giving them.
Are you just giving them a straight-up list of tasks and making them go to an effort to work out specifics?
Are you flagging everything as a high priority?
Are all the tasks complex, and will take a lot of dev effort to complete?
One of the easiest ways to get SEO tasks implemented with a development team, is to become gap-fill work.
Development teams may have a 2-week sprint, and complete the work on Wednesday or Thursday. They will be looking for something quick and easy to pick up, that doesn’t require a full sprint brief.
Have gap-fill tasks ready for them, so that they can squeeze in when they need them.
Make a product/development team’s life easier, and you stand a higher chance of having your SEO work pushed through.
If all else fails – cake.
Legit.
Cake is an awesome way to sweet-talk your way to the top of a task list.
Who doesn’t love cake? And who doesn’t love people that bring them cake?
SEO task prioritisation template
Want a plug-and-play ICE scoring & task prioritisation template?
Grab my SEO task prioritisation sheet with the link below;
A little pre-note here is that the next couple of posts will take a little side-step. Purely because I don’t have my own content, and want to get something out ASAP.
You can do this infinitely better than I will be, if you’re putting in the time or money to build out your dataset.
We’ll be back on track afterwards.
Time to crack on!
Fixing the design
I couldn’t bear it.
Haven’t played with too much WordPress lately so wasn’t sure what design to go with.
Have just installed Astra free and tweaking it a little bit. Much cleaner. Might upgrade it to paid to get a bit more customisation too but it’s already infinitely better than before.
Managed to clean up some of the more “bloggy” stuff, and have also got breadcrumbs going!
Just need to remove that pesky home button but I can sort that one later.
Also removed comments, cleaned up a couple of things and added a pretty header menu.
And by pretty I mean non-default. It’ll do for now.
Will be getting rid of that ‘previous post’, along with the recent post links at some point too. They’ll kill some of the topical groupings by linking to who knows where.
Happy to also say that the change to Astra fixed the main category design problem!
Need to work out how to remove all the posts listed below though, as they’re already linked to from within this content.
Unfortunately, the theme killed the subheadings for the page, but that’s liveable for now. Can add some custom CSS to address this later… I hope.
Upgrading the content
On top of the continuous data expansion via my VA, I need to update these keyword templates.
Let’s be real, they’re shit.
I still might get a writer involved later, but let’s have a play with an AI tool.
Generating some generic template text with an AI tool
My go-to AI content generation tool is WriteSonic – for now.
You can ask it pretty much anything at all, and it could return a full response, or just a simple answer to a question.
Maybe we can just ask it, in bulk, whether an <animal> can eat <food>.
Once registered, you can jump straight into their playground.
I haven’t spent as much time working directly in Open AI & their API, so this project is changing that.
To start, we can just straight up ask it the question.
Yes, a robot in the computer wrote that.
Guess that means the project’s done now, doesn’t it?
Well, I could stop there, and if I was happy to spend the extra money for a VA to build out a quality data set and have a writer build out quality templates, this is definitely where I’d stop.
Alas, I’m not. So let’s take this further.
The initial step was obviously not enough content, so let’s try get that expanded out.
So that’s a great start, and it can generate content for each section we created.
Completely related content.
You can then kindly request two paragraphs of content, instead of a single paragraph by using the “write two paragraphs” prompt.
I am seeing it hit and miss, but seemed to get higher success throwing a ‘separate’ in there.
The ‘please’ is just to make sure the robots don’t come for me when they take over.
That’s 89 words though, from a simple question.
I’m no expert here, and have only just started learning the basic prompts, so feel free to throw anything in the comments that could help out here.
This is using the DaVinci model, which costs $0.06 per 1,000 tokens.
OpenAI says it’s ‘approximately’ 4 characters per token, including both the prompt and the output.
The above prompt was 77 characters and the response was 495 characters, giving us a total of 572 characters.
572/4 = 143 tokens
143 tokens = 143/1000 x 0.06 = $0.00858
That 89 words cost us less than 1 cent.
Let’s say we want to generate 8 sections of similar content for a page, it’ll be $0.069 per page.
To then do this for 1,000 pages, we’re looking at $68.64
Would I do this for a test project where more emphasis is on the process & the overall build and I really just need a tonne of filler content to build out the pages and prove out some concepts whilst still having somewhat shot at being indexed and possibly ranked for longer-tail keywords?
You bet. Game on!
Bulk generating the AI text
Let me preface this by saying I have never done this before. Legit.
This is my first time doing any sort of bulk generation of text directly with an AI model, and not using a tool with an interface and a VA to enter the data.
This. Changes. Today.
Ideally, we get this working directly in Google Sheets.
Would mean we don’t need to worry about jumping in and out of sheets, or doing any bulk imports. As new data & combinations is added, we could just run a formula and boom, we’ve have some magic words.
Input a keyword, and it’ll spit out a title and description. It’ll also cost a little bit due to a rather large amount of sample data hardcoded in the prompt though, so don’t overdo it!
Rather than trying to modify it, a mate shared a mob called Riku.ai.
They’ve been working on a plug-and-play sheets addon, and I’ll be going with that.
I’ve started by building out a little Google sheet to test things, along with estimating the costs of the outputs;
Something I might look at cleaning up later, but this was more of just a way for me to learn how to best write prompts.
I was manually copying and pasting these prompts into the playground to test, and then just paste the output here.
Riku just released the Google Sheets add-on though, and if you’re a paying member (they’ve got a discounted option via app sumo for July) you can find the Google Sheet in their FB group.
Everything runs through your Riku account, and you set up prompts in there which is rather annoying as you need to work in 2 systems to make updates. To get this working how I wanted it I had to create ‘blank’ prompts in Riku, and then just fed in this prompt. So I am essentially just using it for the Google Sheet at the moment.
Loaded it all up, threw in a heap of prompts and this is what happens;
This is magical.
Following this format, I’ve built out a Google Sheet containing all the different prompts I might want to use;
Throwing some of these into the Riku sheet and running the AI over it, we get;
At top level, it looks pretty cool.
I get the exact question format I am after, along with a chonky paragraph.
Problem is, I started to see some issues when looking at the generation in bulk.
Like this one.
As far as my Googling tells me, no, they can, in fact, not.
I know I’ll get a heap of false info for this, but that is why the prompt was tweaked further to become;
So it all comes down to the prompts and some examples of certain expected outputs from the prompt.
I also realised that the prompts I was using around the paragraphs, just weren’t generating anything clean enough.
I figure it’s best to take it back to basics, and just use the dynamic variable system I have already built.
The one that the VA started filling in for me, where we just place some simple text and bulk it out with the generic text.
This is where the true intelligence of OpenAI shines.
The current text system to generate our combo pages accepts inputs of text like the below examples;
So let’s see what happens when I try to request something similar from OpenAI.
Nice summary, but, it doesn’t follow the format.
Well, we can feed the AI some examples of the formats we want it to follow by using these magical stop thingies ##.
Google tells me that’s pretty good.
Turns out the ‘comma-separated’ request might have been a bit confusing, particularly for some of the other prompts.
So instead, I modified it to be ‘a summary’ and the output slightly improved;
This time, it also followed the format in a couple of the examples where there is a full stop mid-sentence.
It followed it PERFECTLY.
Honestly, was not expecting this. Pretty neat.
I’ve fleshed a similar thing out for the other main sections, so that we can run this and get the content.
Unfortunately, I ran into an issue with the Riku Google Sheets.
It wouldn’t allow line breaks sent via the Google Sheets. I stripped them out, and thought I had gotten it to work, but unfortunately it only worked for a few. After that the Google Sheets kept acting like the stops didn’t exist and I had to drop it. It was too time consuming to patch the broken ones.
However, I was sent a direct Google Sheets script that I made work perfectly.
My prompt with examples, line breaks and stop sequences on the left, with the output on the right;
Bingoooooo.
I added a little tweak for some of them, that if the animal couldn’t eat the food then don’t run the request for the prompt.
In this case, it was for the benefit of food.
I don’t think it’s much of a benefit if the animal dies after eating food, and the robots can return some rather interesting stuff, so was best to just ‘ignore’ them for some sections.
It will need some more examples in the future though to account for different possibilities.
I ran this across 150 combo pages to kick things off, and it cost approximately $1.
In total.
$1 for an 80%+ ish, arbitrary, correct data set for all of those responses.
Waheeeeee.
I further tweaked the model to include the text template element that will sit before the text. This way, the robot has something to kick it off and they al nicely fit the template and will read cleanly.
One of the new templates is;
Which, as you can see, outputs exactly what I need. I nice and direct answer.
Yeah, a lot of them are ‘upset stomach’, but hey, that’s what happens with food.
It now works a treat and I’ve added content for them all, for 150 pages.
You’ll probably spot some stuff that might not be correct.
The robots get it wrong sometimes though.
There’s just so much trashy content out there these days that these little guys suck up way too much junk to feed it.
Yes, that food is okay No, that food is poison Oh, I feed it to her as a treat Yeah you should be okay Only a little bit though
There’s just no solid answer for it sometimes, so it confuses the AI when being asked yes or no.
Sometimes, the two different AI models will give two different answers. The same thing happens just using two different prompts at times.
My initial generations are mostly accurate, but I’d say it’s 80%. I’d like to get that closer to 95%.
For certain yes/no questions, I feel like if it’s a no, there’s a higher chance of the no being correct than a yes.
My plan later will be to run both models, and maybe an extra prompt, and then try and pick the average to give it more chances to be correct.
I’ll also extend out the samples, to try and be a little more accurate. It will cost more, but accuracy will be required, particularly in the yes/no question.
Something I’ll test out later, but I’ve got enough for now, ill probably just add a caveat or something as a warning just in case it actually ranks.
Adding an out an ‘about us’ page with AI
We need to now throw up a basic ‘about us’ page that gives it a little bit of a… err.. real feel.
Since we’ve just used a tonne of it already, why not throw a bit more AI in the mix.
Don’t need anything special for now.
Threw in a rather detailed prompt and the playground threw back the below.
It randomly spat out a domain name that is completely open for registration… at the time of posting. Go on. I dare you.
No clue what it did there, but pretty cool!
Added a logo
Just some text and an icon I whipped up in 2 seconds.
It needed it.
Re-upload the data
And just re-uploading all our data.
Updating the old pages, and adding in all our shiny new data.
150 posts, plus the 10 categories (imported separately), done.
Where we’re at now
After the initial build creation, and internal linking setups from the previous posts, we’ve now significantly scaled page creation along with expanded the available content for each page.
The pages are looking pretty text-heavy now though…
Maybe this guy can help us out.
So what does this banana face-hugger cross doggo have to do with the build?
