The Gloebit payment services is back up this week, kind of, after more than a month of downtime.
“We restored the service last weekend,” Gloebit founder and CEO Chris Colosi told Hypergrid Business on Tuesday. “People should be able to access their accounts, buy and sell Gloebits, and cash out.”
However, some issues still remain, he said. “Sending email from the service is still down, so anything that requires email — password resets, confirming an email address, contact-us page — are not back yet.”
There are also some performance issues which are impacting some inworld transactions. he added. “We’re working through items to improve the performance.”
Gloebit is a popular multi-grid payments system, kind of a PayPal for OpenSim. So the outage affected many grids. Equally as upsetting for many was the lack of communication. Statements from the company have been short, and, for the most part, not informative to actual end users.
“We’re back,” Colosi said in a statement on Facebook Monday.
There were no apologies. No explanations of what the company will do in the future to avoid a similar shutdown. No acknowledgement even that it has hurt the users of the service.
During a month-long outage, Colosi has posted eight statements on Facebook, nothing on Twitter, no updates on their website, and no direct contact with the users affected by the loss of the service.
The first note was short and uninformative.
“We are experiencing an outage,” Colosi posted on April 6. “AWS believes our service is up but it is not accessible. We have a support ticket into them.” AWS — Amazon Web Services — is the platform where the Gloebit application and data are hosted.
This was followed by a few short updates over the following four weeks with technical details about the company’s problems getting support from Amazon. There were a few details about how the service is configured, but no information about impact to users, or what the company was doing to mitigate the effects of the outage.
For example, on April 11, Colosi wrote, “We haven’t yet found a way to restore or recreate our old elastic beanstalk environment. We are working on dockerizing our service to remove the dependency on elastic beanstalk and allow us to easily deploy it elsewhere. This outage and the one in December were both caused by elastic beanstalk, so we looking forward to getting off of EB and hopefully restoring a more robust service.”
He was referring to the AWS Elastic Beanstack, an application deployment platform offered by Amazon, and to Docker, which is a way of packaging applications into small virtual containers that can be moved from one cloud to another. While of possible interest to other application engineers, these updates were of little use to actual users of the platform.
Meanwhile, even though Gloebit is supposed to be up, some people are still having issues.
The Dynamic Worldz grid still has not seen the service restored, grid owner Danny Cruise told Hypergrid Business. However after further research, the problem only shows up on grids hosted on Windows servers and, more specifically, Windows server 2012 and earlier.
“As I have a mix of servers, all on different versions, I will try and move any of our Gloebit enabled regions onto the newer servers,” he said. “This is a .Net issue and Windows 2012 and earlier do not support the required release.”
Grid owners, users are furious
OpenSim grids use Gloebit to provide payment for their virtual worlds in a way that bring in revenues — they get a cut of all in-world transactions — but removes risks for themselves. Avination, for example, once one of OpenSim’s most successful commercial grids, lost $126,000 to hackers, which contributed to its demise.
For merchants, Gloebit is an easy way to accept payments from stores on multiple grids, and have it go into one centrally-managed wallet, kind of like a PayPal for OpenSim. Instead of trying to figure out how to cash out multiple local currencies, they only had to work with one, relatively easy, online interface.
And for OpenSim residents, Gloebit is a way to keep money safely in an online wallet that follows them from grid to grid to grid. If one of those grids goes down, all the money is still safe — just like your PayPal balance isn’t affected if one of the websites that uses it for payments goes out of business.
If anyone could get a payment system like this to work, it should have been Colosi, who’s best known as the guy who used to run the Second Life Marketplace and the Lindex exchange. Today, Colosi works as a software engineering manager at financial technology firm States Title.
The recent outage — and the series of outages last year — are giving people second thoughts.
“Back when I worked at IBM we used to say that if a bank or financial institution did not recover from an outage within 24 hours, it would be out of business within a year in 99 percent of the cases,” said Xmir grid owner Geir Nøklebye.
Technical issues are sometimes out of your control, said DigiWorldz grid owner Terry Ford. But constant communication with customers is vial.
“Even if the communication is not what they want to hear, at the very least your customers will know someone is aware, has taken responsibility, and they are working towards a resolution,” he told Hypergrid Business. “In the case of the Gloebit outage, many users felt as if they were being avoided, wondered if they would ever get their money, and were upset they weren’t being updated frequently. I believe if you are to provide a service which so many users depend on, you need to have plans, systems, and resources in place in order to resolve all issues, from minor to major. It’s clear in this case, if such a plan existed, it has failed miserably.”
DigiWorldz give users the option of using Gloebit or the local DigiWorldz currency on their regions.
Many users have already asked him to remove Gloebit, he said.
“Some are saying they would return, and others saying they would never use the service again,” he said. “Which is similar to what I’ve heard from other grid owners and users regarding this outage. Hopefully, the lessons learned from the outage will help to make the system more reliable, and will help to mitigate any future issues, but for now, I think many users are lost forever, while others will be willing to try again. I believe if you are to provide a service which so many users depend on, you need to have plans, systems, and resources in place in order to resolve all issues, from minor to major. It’s clear in this case, if such a plan existed, it has failed miserably.”
AviWorlds, OpenSim’s third large grid by number of users, has already switched away.
“We held in-world meetings with our staff and also polled our residents and we have decided to make a switch to Podex,” AviWorlds CEO Josh Boam told Hypergrid Business. “It is not acceptable for a business to operate like this.”
Neverworld, which converted the entire grid to Gloebit a year ago after four years with Podex, has switched back to Podex.
Even before this outage happened, the grid wasn’t happy with Gloebit, grid owner Govega Sachertorte told Hypergrid Business.
“They were very slow in answering any support emails that we sent to them or in providing any assistance,” she said. “We will still offer it for personal regions if they absolutely insist in having it but I give Gloebit a big thumbs down.”
German World Grid is staying away from virtual currencies of any kind, grid owner Mickie Bohannes told Hypergrid Business.
“Both of the options that exist seem too insecure to me,” he said.
Plus, a virtual currency makes less sense in a community where people want to have content shared for free.
“Despite that, I think it’s a great thing that there are such currency providers,” he said. “My goal is to strengthen OpenSim and to keep people in OpenSim.”
Podex is the single biggest alternative to Gloebit, but it uses a different operating model.
With Podex, each grid still has its own currency, but Podex provides exchange services to make it easier for residents to buy and sell that currency or to trade it for currencies on other grids. Shopping is much less convenient, though, since merchants have to manage multiple currency accounts and shoppers who teleport to a new grid will have a hard time making purchases there. Gloebit was perfectly positioned for hypergrid shopping — when the service was working, of course.
Another option is to go fully independent, creating a currency system from scratch. That leads to potential security risks, if, say, users buy currency with stolen credit card numbers, redeem it for cash, then the credit cards are cancelled and the payments are reversed, leaving the grid in debt.
That risk is what keeps AnSky from setting up their own currency, said AnSky grid owner Andrew Simpson. As a result, he said, he’ll be sticking with Gloebit — for now, at least.
“Until I can find a better one or bite the bullet and take the risk of using our own home-built system for currency,” he told Hypergrid Business.
Meanwhile, for region rentals, customers must pay with PayPal, he said.
AviTron will also continue to use Gloebit, but is actively looking for alternatives, said grid owner Alexsandro Pomposelli.
“JOpenSimMoney is great, but it is a paid one,” he told Hypergrid Business. “It charges a per sim amount.”
Another option is PayPal.
It has high costs when used for micropayments, but most grids already use it for things like region purchases.
“I am intending to use PayPal if needed,” said Kashi Takeshi owner of the Virtual Worlds Zone grid. “Gloebit proved to be too unstable to function as a trustworthy money platform for OpenSim.”
“Virtual currency solutions come and go, and OpenSim has had multiple providers over the years that gained traction on multiple grids but ultimately suffered the same fate,” said Ilan Tochner, Kitely’s co-founder and CEO. “My recommendation for people who wish to buy or sell content without having to rely on the continued operation of such virtual currency systems is to do it directly using real money via their PayPal account. You can use PayPal directly on Kitely Market or using several third-party inworld vendors that support PayPal payments.”
People can also accept donations or rent land using PayPal, he added.
To make it easier for in-world merchants to set up payment systems, there are products available on the Kitely Market.
Ocean Engineering‘s PayPal Rental Box and PayPal Donate Box handle the payments processing, but at a cost of 20 to 30 cents per transaction. Plus, with PayPal, the payment recipient is notified of the payer’s email.
Another option that merchants can consider is sending buyers to the Kitely Market website for the final purchase, said Tochner, by, say, having their product sale sign link directly to the item’s market listing.
“That way when the sign is clicked the relevant page will open in the buyer’s browser and they’ll be able to securely complete the order from there — including in-world delivery using the Kitely Market hypergrid delivery system,” he said.
The process does add a few extra steps to the purchase process.
On the other hand, it also helps merchants serve customers on defunct grids, by allowing them to re-deliver the purchased items to customers’ Kitely avatars, said Tochner.
And there’s also a privacy benefit, he added.
“When people use PayPal on Kitely Market the buyer and the seller don’t see each other’s PayPal information or email,” he said. “Instead, they only share their avatar name, just as they would if they made the transaction in-world.”
Zetamex is sticking with Gloebit
“The recent outage was vastly mishandled, there is no doubt about it,” said said Vincent Sylvester, CEO of Zetamex Network, an OpenSim hosting provider and grid operator.
But it’s still the only game in town, he said, since the alternatives are less flexible and less secure, he told Hypergrid Business.
“Seeing the ease at which Gloebits can be sold indicates residents across the metaverse are still purchasing now that the service has returned,” he said. “As we have in the past, we will continue to work closely with Gloebit to aid them in strengthening their service and improving the quality for users.”
As frustrating as the outage was, it illustrates the inherent problems of using Amazon, he added. The company doesn’t provide adequate support and solutions for issues on their network.
And aside from the infrastructure issues, Gloebit’s design and setup, while still needing work, is much better suited for actual commerce than the alternatives.
One of the most popular option for grids creating their own currencies, for example, is the DTL money server for OpenSim, he said. “The disclaimer of ‘play money’ is as prominent as ever and rightly so.”
In addition, the OpenSim user base is forgiving.
“The metaverse has seen grids go down for longer and still somehow retain the majority of their userbase,” he said. “Commerce and money is not the essence of life, thankfully.”
Zetamex Network was one of the launch customers of Gloebit he said. “We continue to work with them to resolve issues and find a way forward for the service.”
He’s not the only who’s betting on Gloebit.
As the owner of an independent, non-commercial grid, Phantom Rose, I find Gloebit funds very valuable to me,” said Phantom Rose grid owner Lannorra Sion. “It really is the currency for the metaverse. It allows anyone from any grid to buy things from different creators from different grids. I use it exclusively. If Gloebit does not come back, my only source of buying items I want is through the Kitely Market. Which would be very disappointing. Kitely Market is great, and I do buy often from them, but it doesn’t carry everything available.”
With Gloebit, she can go shopping at pop-up markets on Discovery Grid, for example. “Many of the merchants will use Gloebits and I will buy things that I have not seen on the Kitely Market,” she said. “I will definitely be using Gloebit once their service is running normally again.”
She has been burned over the years when grids shut down, however, which is one reason that, in addition to owning small bits of land on other grids, she also has her own grid.
“Going forward, I think I will buy Gloebits as I need them, and not keep much as a balance,” she said. “But otherwise, I look forward to using their service again. And I hope they stay in business!”
“I plan to stick with Gloebits,” said Larry Schrupp, owner of the Langalf’s Demesne grid. “There are only a handful of users on my grid, and no real commerce at present, so the down time has been a nuisance only. For us it is more an experimental environment. We use Gloebits in transactions on other grids, and it does not hurt me to support it on my grid. I do not really know of any viable alternatives for a small grid like mine.”
“It is good to see them back, I hope they can go forwards now and hopefully get stronger in the future,” added Enchanted Grid owner Dai Rhys-Owain.
Source: Hypergrid Business