Zach: Welcome to another episode of the Woo Dev Chats. I’m Zach Stepek, here with Carl Alexander today.
Zach: How are you doing, Carl?
Carl: I’m doing good. How about yourself?
Zach: I’m doing really well. We’re missing Till today. He injured his knee a week ago. That’s not fun. But he is hopefully doing better. He said it’s mostly healed now, which is nice.
Carl: So he is taking it easy, as one should.
Zach: Yeah, he absolutely should take it easy. So today we’ve decided that we’re going to talk about the evolution of hosting and what does that mean?
Carl: Yeah, we’re two people that work with hosting. You’re with Cloudways, I’m trying to build my own thing. I think it’s a really interesting discussion. So there was also Syed’s tweet thread. Did you see that a week ago?
Zach: I did not.
Carl: So he did this tweet thread, we’ll definitely link it in the description, but he was discussing a bunch of topics with WordPress, but one of them was basically managed product hosting. So like managed EDD, which they’re doing with Site Ground. Nexcess is basically doing managed LearnDash Cloud. So just discussing this idea of where hosting is going with more dedicated hosting platforms. So it’s not just WordPress now, Oh, I want to use this specific product and I want to host it on a platform that’s really optimized for this specific product, and how do we that? So I thought that’s really fascinating because I feel that’s where, and I’m curious to hear your opinion on that as well, I feel like that’s where it’s going for the next decade.
I did a Reddit post two weeks ago, I think, discussing how basically hosting WordPress sites, your news sites, your content sites is generally pretty solved at this point. There’s a bunch of different architectures, but they all have similar components, and they’re just different flavors of vanilla ice cream, basically. Do you like this vanilla flavor? Do you Häagen-Dazs or do you want just standard vanilla flavor? There’re just pretty much different ways of doing the same thing. So hosting WooCommerce is just starting. Well, Nexus has been in it for a while now, for a few years, but GoDaddy, at WordCamp US, just came out with managed WooCommerce hosting and I think WP Engine has it already as well, I think. And then there’s some that are more dedicated.
Zach: Bluehost announced something during WordCamp US as well, and that’s really interesting to see where that stuff is going. I’m really excited about the GoDaddy offering, but it’s Pagely and SkyVerge. I believe Becca ran this team to build this new product. So Beka from SkyVerge.
Carl: I don’t know who Beka is and I don’t even know really much about SkyVerge, so talk away.
Zach: Yeah, so SkyVerge, for those of you who don’t know, built most of the plugins that are on the WooCommerce marketplace. So they have been an integral part of the success of WooCommerce by making it do things that it didn’t do by default. They wrote the structure for how payment gateway plugins are created and created over 70 payment gateways themselves. So a lot of those are them.
Carl: That’s an insane amount of payment gateways.
Zach: It really is. I might be wrong on that, but it feels like 70. So if I’m wrong, absolutely call me out. But it feels like a lot of payment gateways, and 70 is just the number I’ve always gravitated toward. But there are so many plugins that they’ve built. WooCommerce memberships is them. They built the Stripe plugin before it was handed over to the WooCommerce team. They built the Square plugin before it was handed over to the WooCommerce team. And so GoDaddy acquired SkyVerge a little over a year and a half ago now, I think. So in doing so, they acquired that entire library of plugins that are in the marketplace. So what this offering from GoDaddy has done is it’s taken all of this knowledge that SkyVerge has and it’s coupled all of those plugins with a hosting platform that’s built on top of Pagely’s infrastructure. It’s a really cool offering. So you get all those plugins that normally you’d pay a hundred dollars a piece per year or $50 a piece for some of them per year, and they’re just bundled with your hosting. So really pretty cool.
Carl: Yeah, I think that’s what a lot of players are looking at doing now. That’s why I think a lot of the hosting companies are buying all these plugins is they’re trying to build their wall garden offering. How good it is for the larger community is up to debate. But as somebody that’s building something to host these WordPress applications, as I like to call them, because they’re not really WordPress sites, they’re more applications than content sites. It’s really hard to optimize and increase performance and things like that when you can just install whatever plugin you want. Especially with Woo, I’m sure you’re well aware, optimizing Woo with a bunch of add-ons and things like that is really, really complicated.
Zach: It can be, especially if some of those plugins or extensions are written well, and some of them are written really badly. And I’m not saying they’re badly written in general, I’m saying that they’re not written for scale. I have to be really clear about the delineation there because I’m not trying to be rude to people who’ve built good plugins that just don’t scale. It’s just that it’s really hard to test for scale, and especially toward the beginning of the ecosystem’s creation, it was even harder, because nobody knew what scale was going to look like in WooCommerce. Right?
Carl: And e-commerce scaling is just hard period.
Zach: It is. Yes.
Carl: Coming from Magento and stuff like that, it’s always hard. So it’s not also WooCommerce’s fault necessarily. They have some tech debt that they really need to be working through using the post meta. Now they have dedicated tables for that, for example. That’s a good example of tech debt that they’re slowly going through. But that applies to plugin as well. Some plugins just don’t scale well because they just weren’t designed that way initially, but now they’re starting to be used on stores that are going to have a lot of traffic and a lot of sales volume, and then all these things break.
Zach: Yeah. Well, and I think that’s the same for any of these larger ecosystem says Jonathan Wold likes to call them. Anything that introduces a suite of functionality for an audience, anything that provides an integration layer for additional extensions, and that creates and shapes its own ecosystem over time, that’s an ecosystem plugin. That’s how he defines it. And I think that what we’re seeing is that hosting companies are seeing an opportunity with these ecosystem plugins to own a part of the ecosystem. So Automattic acquired WooCommerce to own WooCommerce’s ecosystem. That’s the long and short of it.
There’s a benefit to having an ecosystem like that. WP Engine wanted Advanced Custom Fields because it’s an ecosystem. It’s something that other people have extended that is integral to the way that some agencies do their work. And all of these ecosystem plugins, they have a marketplace around them. They have other companies adding into what they can do. Gravity Forms is an ecosystem plugin because other people create plugins for Gravity Forms. Gravity View wouldn’t exist without Gravity Forms.
Carl: Some businesses with multiple employees, sometimes dozens, just build Gravity Form products. That’s insane.
Zach: It really is. But these ecosystem plugins are really the big hosting problem as well. Right? And I say problem, but really it’s just challenge. Right? They’re a challenge to host well. The bigger the ecosystem, the harder the challenge is. And one of the interesting things about having this open web ecosystem that we live in, in WordPress is hosting providers. Hosting providers, they’re a part of this ecosystem. They benefit from the ecosystem, but they’re really free riders and this whole free rider problem and open source is an interesting thing. WordPress is free to use. Right? The four freedoms guarantee no restrictions on its usage. It’s maintained by volunteers, and the confidence of free riders and WordPress has led to investment.
So that’s why we have ecosystem plugins. Hosting companies should be investing in ecosystem plugins and extending their distribution. I fully agree with what Jonathan had to say in his article on hosts and the free rider problem regarding that. And the other thing he says is that product creators should be working hard to build partnerships with as many hosts as possible and not be exclusive. And I wholeheartedly agree with that too. So WooCommerce has done a really good job of this. There’s no single host that is the defacto standard recommendation for WooCommerce hosting. There are some that are better than others.
Carl: At least not yet.
Zach: Right. Not yet. It could happen in the future where there’s somebody who pulls far ahead of everybody else. But the great thing about a marketplace like this is that if somebody pulls ahead, that helps everyone. Right? Because everyone else gets to see and learn from what that person did, or what that team did. And so that’s a very interesting piece of this whole puzzle. And we’re seeing some really interesting things happen at WordCampUS. They announced WP Cloud.
Carl: Apparently it had been existing for over a year. I only learned about it a week and a half before WordCamp US. But if you look back, I think on the way wack machine or whatnot, I think it’s from 2021, it came out, the site’s been up.
Zach: Yeah, it’s been around. And it looks like Jessie Frick is involved.
Carl: She’s with Pressable? Right?
Zach: I believe so, yes.
Carl: Yeah, because Pressable runs on that. So that’s what they told me.
Zach: And the interesting thing is WP Cloud feels a lot like what VIP Go was. Right? Now we have WP Cloud instead of VIP Go.
Carl: I mean it’s very close to what Ymir is because Ymir is basically an API product. So they’re very similar. I was just like, oh okay, I’m not the only one basically doing API driven infrastructure anymore.
Zach: No, it’s a validation of the work you’ve been doing with Ymir, right?
Carl: Yeah, I have an interesting take on some of this because I think you’re right, for some aspects I think it’s going to go that way. But I’m also thinking, I think a lot of those plugins are going to become their own standalone products.
Like Gravity forms because have Typeform. Right? Technically if Gravity Forms wanted to just self host on a platform, that’s where I think I come in actually, but if Gravity Forms wanted to just host Gravity Forms, not WordPress, not nothing, just host forms based on WordPress, and you can just create forms and try to compete with Typeform, they could do that if they could manage an infrastructure to just run their product on.
So I think there’s a lot of that as well that’s happening too. So you have this dedicated hosting setups. So managed WooCommerce, maybe something for LYFT LMS, because I think LearnDash Cloud is more what I’m thinking about with Gravity Forms. I think it’s just a standalone.
Zach: I would agree with that. Yeah.
Carl: I think it’s just standalone, but that’s what I mean. But that’s a good example of the dichotomy of the two things that are going to happen, I think, is you’re going to have dedicated hosting services for specific products, but you’re also going to have basically the standalone WordPress Plugin Pro as a product in itself, like LearnDash Cloud is. And I think that’s also a thing that’s going to probably happen because some of these products could compete, theoretically, with those SaaS products. The problem that they have right now is that, how do I host that?
So LearnDash was easy because they got bought by Nexcess. It’s a StellarWP brand, so they’re under Nexcess. So Nexcess was just like, okay, we’ll just build the infrastructure for that. But if you’re on your own, how do you build that infrastructure? So that’s where I’m interested in that space, obviously, because I think there’s a lot there as well for these larger platform ecosystem plugins.
Zach: I agree. I think that what we’re going to start seeing is, well what we’re already seeing in the WooCommerce space, which is people building these customized hosting products just around WooCommerce. You said I’m working with Cloudways, obviously. There’s a WooCommerce stack at Cloudways that you can deploy. We have players that are focusing on just WooCommerce hosting like Convesio. Nexcess has been around for a while. There’s just a whole bunch of players in this space that are trying to make WooCommerce better, and make it easier for store owners. And that’s really the key. Right? Because we can’t have an army of people running agencies that are making tons and tons of money off of customizing WooCommerce to make it scale. It’s just not something that can be sustained long term. Customizing WooCommerce to build stores? Absolutely. That’s something that long term can be sustained. But working just in this performance and optimization space, it’s something that, at some point, the community or the ecosystem itself needs to take care of. Right?
Carl: Yeah, it’s hard. My friend Patrick, who’s doing Woogo stores, basically on top of Ymir, is basically doing something like that where he is trying to basically build Shopify. Because there’s nobody really that’s trying to do it right yet, something like Shopify, but with WooCommerce. So you basically sign up and you get a store. Just the store not at the entire WooCommerce install or whatnot. You don’t even necessarily know that it’s WooCommerce, you just set up a store and then it’s WooCommerce behind the scenes. How do you get to that? How do you get to scaling that? Because right now that’s a thing I was saying, nobody could host Kim Kardashian’s clothing line with WooCommerce. She’d send an Instagram out, for a sale, and it would just literally blow up.
Zach: Well, to be fair, at one point Kylie’s cosmetics line was on WooCommerce and it was running well. It wasn’t perfect, but it was running well and making tons and tons of money. But it did eventually move to Shopify. And why did it move to Shopify? Well back then, this was 2017, I believe, 2016. It moved to Shopify because they were sold on the fact that Shopify would always have the performance there and they wouldn’t have to worry about it anymore. Right?
Carl: Yeah, exactly. And they probably sell even more now than they did back then. So think of the problems they were having then versus now. That’s what I mean. It’d be very hard right now to sell a really, really high volume, burst volume WooCommerce site. Just keeping it up would be really hard. Borderline, I’m not sure if possible.
Zach: It’s not impossible. One of the things that we did when my former agency and I were working on native deodorant, we would work on elastic beans stock, have auto scaling in place, but we’d also prescale. So we let them prescale the number of servers that they had for horizontal scaling of their web heads before they sent out a marketing email. And that generally was enough to keep the site up with a two million person email marketing list. And they did really well on Woo. In fact, Moise, the founder of Native at e-Commerce Fuel live in early 2020, I believe it was January of 2020, he said that he wished he had never left WooCommerce.
Carl: Who is he with now? Shopify?
Zach: So they’re with Shopify, because they were acquired by Proctor and Gamble for a large amount of money, and all of Proctor and Gamble’s properties are on Shopify. So they moved to Shopify, but they lost some of the functionality that they had on WooCommerce, that was specialized and built for them. The ability when somebody went to cancel a subscription, to put the questionnaire in front of them asking them why they were canceling, and move them into a lower tier product rather than having them cancel completely. It was all custom built for them on top of WooCommerce subscriptions, and couldn’t be done, at the time, on Shopify because none of the subscription platforms had that. So their rate of attrition went up.
Carl: Yeah. Those are good examples, and that will be the problem also if somebody builds a WooCommerce platform, you’ll be locked into whatever plugin suite that they want. You’ll be like, okay, well we optimized for those. We’re not going to let you install whatever you want because we don’t know how it’s going to behave. Because, like you said, to loop back to what you said at the beginning, it’s not necessarily the plug-in developer’s fault, but there’s just these scaling issues that you can’t really grasp until you’ve seen it.
Zach: Let’s be fair, even WooCommerce didn’t know that WooCommerce was going to scale to the levels that it has. So when this whole thing started, nobody expected that WooCommerce was going to be powering stores that were doing hundreds of millions or billions of dollars in sales per year. It just wasn’t something we thought was going to happen. And then it did, and then it broke, and then we had to figure out to make it work. That’s been the challenge, building for scale is much harder than building just for functionality. Right?
Carl: Boy do I know that. That’s all I deal with.
Zach: And so it presents unique challenges. How do we handle things like fragment caching? How do we handle things like optimizing our queries to make sure that they are running as quickly as possible? How do we create adequate indexes so that look ups happen faster? How do we make sure that our code is running in the most efficient way possible? How do we get away from the fact that at scale a structure of the value and value meta, post and post meta, or item and item meta, doesn’t work at scale? It just doesn’t work when everything is a custom post type, it just blows up.
Carl: Magento has the same problem. It’s a known pattern. It’s called EAV, it’s Entity Attribute Value. And basically that’s what post meta is. There’s a trade off, basically. You get flexibility in how you can basically assign metadata to things. You can assign anything to a post, you can create whatever post meta and sign whatever information you need to. The problem then becomes, okay, well that basically makes the database incredibly hard to query, because now you have to basically do a whole bunch of joins.
Sorry, I’m being a nerd right now. Zach’s head is nodding vigorously, but basically after that you’re stuck doing a bunch of joins on the same table to get the information you need on the same entity. And that’s where the whole thing breaks down, because when it’s all on the same table, it’s a lot easier. You just look up the post and then you get all the information you need. The problem is, it’s hard to adapt that table because you want it to be flexible with the data that it has.
So it’s managing that trade off, that’s really challenging, which is not really WooCommerce’s fault per se, but it’s a common e-commerce problem because e-commerce suffers from the problem of you need a lot of custom data. If you sell shoes, you need to keep shoe sizes, manufacturers, that’s not just SKUs. If you sell coats, it’s something else. If you sell cosmetics, it’s something else. So you need something, a data structure that you can with with to represent this information. And that’s even more complex if you’re a hardware store. Now you have to deal with all sorts of different things that have all sorts of different metadata. And you have to be able to query that and get that information out, is just very complex.
Zach: Imagine a grocery store with a hundred thousand plus products. Right? I’ve never dealt with that. Not even once.
Carl: Yeah. No, that’d be crazy as well. Large department stores like, RIP Sears, but Target things like that, like you said, it’s the worst of all the worlds. You carry food, you carry hardware, you carry electronics, you carry everything.
Zach: And then add the complexity of market based pricing and market based tax rates. Right? And then you add in the complexity of things like local pickup and delivery and shipping.
Carl: Shipping rates is fun. If we had my friend Patrick, that’s what he used to work on before he started the Woogo stores.
Zach: It’s an interesting thing.
Carl: Oh my God. Yeah, he was ripping his air out. And we’re in Canada. I’m bad, I don’t even know how many provinces we have, but we have a lot less than 50 states, I’ll tell you that much. So shipping rates between states, from state A to state B is maybe not the same from state C to state B. So it’s a hot mess.
Zach: Well, and taxes in the United States are interesting because there are some areas where, I believe it’s in Louisiana, that each parish can have a different tax rate.
Carl: Oh, I didn’t know that. But I love Louisiana because they have parishes and it’s very French, so I love it.
Zach: And there are hundreds of parishes that you potentially need a different tax rate for, and that’s why companies like TaxJar and Avalara exist, because these are complex problems. Right? And so e-commerce, in and of itself, WooCommerce, especially in the context of what we’re talking about, these products handle very complex data, very complex information, and they have to handle it for every possible scenario.
And so we see the WooCommerce team moving toward custom order tables, which we’ve talked about before. We see the need for flattening some of these data types away from the posts table in order to improve overall performance. And some of these things are just now beginning to be tackled by the WooCommerce core team. And hosts have been pushing for this for a long time. When we were first working with Liquid Web, when I founded my previous agency, we were working on the custom order table plugin. That was the first project we did with them, was working on custom order tables, and then working on some crazy, big, elastic stuff that we were trying to make work.
Those were our first projects with them, trying to make WooCommerce faster by implementing new things. And we were running into the problem that they outlined, in this new post they released today, about high availability WordPress hosting, and talking about high performance order storage. So one of the things they talk about, and I know we want to keep some of this stuff for another one of these podcast episodes, but one of the things that they’re talking about is that there’s this overhead that they’re going to keep synchronization so that every transaction is synchronized between both the current post and post meta model of storage and the new data store, which is the WC orders table, and the related tables for order addresses, operational data and order meta. The reason why they say that this whole synchronization thing has to happen is because there are a number of plugins that still aren’t using the CRUD layer that was introduced in WooCommerce 3.0 in 2017.
So the CRUD layer was released in WooCommerce 2.7. The first post about it was October 27th, 2016. For those of you who don’t know, CRUD stands for create, read, update and delete, or create, retrieve, update and delete depending on who you talk to. They implemented these CRUD classes, they’re abstracts that abstract away how WooCommerce interacts with data stores for products, customers, orders, order items and coupons. So since 2016, we’ve been capable, and since release in 2017, we’ve had the code available to implement custom data stores for all five of those data types in WooCommerce. The problem has been that numerous plugin developers still find it easier to interact directly with the post meta table than to use the CRUD classes that were created to avoid the problem that’s being caused by interacting directly with the post meta table.
If you’re interacting directly with post meta and data moves out of post meta, your plugin will break. If you use the CRUD classes and data moves out of the post meta table, your plugin will continue to work. It’s the whole reason they’re there. And if you’re not using the CRUD operations, the REST API uses them already. The admin and WooCommerce uses them. There’s a whole bunch of features that use these CRUD classes. If your plugin is not using them, you are breaking the capability for WooCommerce to move forward into custom tables. I’m just going to be blunt about it.
Carl: There you go. You’ve been shamed. You’ve been shamed.
Zach: You are stopping our forward progress as a platform. So if you’re plugin developer, you’re building commerce plugins, make sure you’re using the CRUD classes. If you need help, reach out. Happy to talk to you about how they work, and what they do, and where the documentation for them are. But they’ve been around for a while now and we should be using them. So I’m going to get off my soapbox now.
Carl: That’s funny.
This content was originally published here.