Amazon Web Services + Microsoft = My Dream Platform

We're implementing some changes to move our processing and file storage components to Amazon Web Services (more on that later). We're using S3 for file storage, EC2 for report processing and SQS for report queuing. The end result will be a very exciting, cheap, virtualy infinitely scalable platform that will remove a lot of day-to-day maintenance headaches and gives us cost predictability.

But I have to say, the lack of support for Windows OS on EC2 nodes is making the transition very difficult. Sure, there's a hack out there to run a virtual Windows instance on top of your Linux virtual host, but it's really slow and it's still up to you to maintain your windows licenses to match actual use. If you need 50 Windows nodes for 2 days, well, you're out of luck.

We're a .Net shop. Our only option is to convert our heavy processing code to Java and run it on Linux. Sure, we could use Mono, but .Net support is limited, and I'm a little concerned about the future of the Mono platform. Unless they get significant support from MS, I just don't see it keeping up with .Net framework advances. I mean, can you imagine Linq support in Mono anytime soon? I can't.

Since they're using Xen in the backend, it seems to me that the only thing preventing Amazon to support Windows is a licensing issue. I don't understand why MS wouldn't make a deal with them. It doesn't sound that complex. Average # of Windows Server instances / yr * volume price. Amazon has the metrics needed. It would require initial investment in purchasing a large set of licenses though, but I don't see why MS couldn't help them out and charge after the fact rather than up-front. And charge the cost back to the end-user.

Assuming Amazon pays $800 for each Windows Server Standard License. Now let's assume they want to cover that expense within the 1st year. That's $.09 / hour, so let's say $.10 is Amazon's actual cost for the first year, $0 afterwards. Charge end-users $.20/instance/hour extra for windows boxes And make a healthy $.10/hour profit during the 1st year, $.20/hour afterwards.

Pricing now becomes

$0.30 - Small Instance (Default)
$0.60 - Large Instance
$1.00 - Extra Large Instance

So you now pay roughly $720/mo for 15 GB of memory, 8 EC2 Compute Units (4 virtual cores with 2 EC2 Compute Units each), 1690 GB of instance storage, 64-bit platform Windows server, with EC2 platform flexibility.

Would I pay this much? Heck yeah. Giving me the power of Windows on EC2 is definitely worth the price, and now I don't have to worry about how many Windows licenses I own. And this could be a cash-cow for Amazon, with very low risk.

So why the heck this is not offered today is beyond me.

TFS 2008 upgrade process

After using Team Foundation Server (TFS) 2005 for a few months now, we recently ran into the well - documented merge bug: "TF14087: Cannot undelete 'filename here' because not all of the deletion is being undeleted.". This was the second it happened to us, and resolving this conflict is quite difficult (More info about this issue). We found a hotfix that might fix the problem, but since we were about ready to upgrade to VS 2008 and TFS 2008 supposedly provided a lot of improvements around merging, we decided to go ahead and upgrade TFS to the latest version. (note: supposedly VS 2008 works fine with TFS 2005 - unless you absolutely need the upgrade, I'd recommend sticking to 2005 until you can free up a few days dedicated to the TFS upgrade).

After reading the install guide and following to the letter, I started the install process. Sure enough, I ran into the following obscure 32000 error:

The Commandline '"C:\Program Files\Microsoft Visual Studio 2005 Team Foundation Server\Tools\TFRSConfig.exe" /setup /install /s "servername" /u "NT Authority\NetworkService" /buildInIdentity /l "1033" /verify /ignoreExistingIISArtifacts /instancename "MSSQLSERVER" /appPoolName "ReportServer" /reportServerUri "http://oceg2/ReportServer" /reportManagerUri "http://oceg2/Reports" /h "servername"' returned non-zero value: -2147022676.

I looked it up, found a couple of posts about similar issue, but no real answer. Looked at the install logs (%Temp%/VSMsiLog****.txt), and found this:

Deleting virtual directory Reports for Report Manager...Deleting virtual directory ReportServer for Report Server...Deleting application pool ReportServer...
Creating application pool ReportServer...
Creating virtual directory Reports for Report Manager...
Configuring Report Manager virtual directory Reports to use application pool ReportServer...Error: 2147944620.Microsoft.TeamFoundation.Admin.ReportingServices.ReportingServicesWmiException: Exception of type 'Microsoft.TeamFoundation.Admin.ReportingServices.ReportingServicesWmiException' was thrown. at Microsoft.TeamFoundation.Admin.ReportingServices.WMIManagementObjectWrapper.PostProcessResult(ManagementBaseObject resultObject) at Microsoft.TeamFoundation.Admin.ReportingServices.WMIManagementObjectWrapper.InvokeWmiMethod(String methodName, Dictionary`2 parameters, String resultKey) at Microsoft.TeamFoundation.Admin.ReportingServices.ReportServerWmi.SetReportManagerIdentity(String appPoolName) at Microsoft.TeamFoundation.Admin.ReportingServices.ReportingServicesConfigurator.ConfigureWebServiceArtifacts() at Microsoft.TeamFoundation.Admin.ReportingServices.ReportingServicesConfigurator.Run() at Microsoft.TeamFoundation.Admin.ReportingServices.Program.Main(String args)
Configuring SQL Server Reporting Services failed.

After fighting with this problem for 4 hours, I gave up and used one of my free MS tech support incidents. I was told by the first person I talked to that TFS 2008 was still in beta (this is as Jan 7th 2008), and therefore the product was supported (??? - TFS 2008 was released with VS 2008 late Nov). After arguing for 5 minutes and speaking to a manager, they redirected me to the VS 2008 install tech support team.

One hour on the phone later, the VS guy told me the issue had to do with my Reporting Services install, so he'd need to get me in touch with the Reporting Services team. I then spent the rest of my day (another 7 hours - I'm not exaggerating) on the phone with that person. By the end of our call, she couldn't figure it out, we had made absolutely no progress and we had to reschedule our call to the next day. I did manage to get this gem though: "You're not the only one with this problem, it happens from time to time, but we usually have a hack around it by bypassing the RS configuration manager and setting the application pool directly in IIS. It looks like TFS assumes Reporting Services can successfully configure itself, which it often can't". Nice one.

So after an 8 hour phone call and another 1/2 day wasted on this thing, I was stuck with a broken TFS install (nothing was working at this point, not even the old version). I gave up on MS and went back at it myself - couldn't spend another day on the phone and see no result whatsoever.

I uninstalled SQL Server Reporting Services, re-installed it, re-applied SQL Server SP1, and re-configured the new instance. Tried re-installing TFS 2008, and everything worked smoothly until... a new error came up, asking me to check my SharePoint/IIS/ASP.NET config settings. I messed with this for about 90 minutes, then decided to re-install SharePoint. I uninstalled SharePoint 2.0, downloaded 3.0 with SP1 and installed that. Of course, I hadn't read the part of the TFS install guide that says "install SharePoint as a web farm", so it didn't work. Uninstalled and re-installed, this time following the directions to the letter.

This time - lo and behold - it worked and upgraded my existing projects successfully. I can connect VS 2008 to my new TFS install, everything works fine. And yes, the merge error is gone, I was able to successfully merge all my branches.

So, long story short, if you're about to upgrade TFS from 2005 to 2008:

• Prepare yourself for a major headache.
• Read the install guide to the letter. Take a break. Read it again. Every word of it.
• If reporting services cause problems, don't waste your time trying to fix it. Even MS Tier 1 tech support engineers couldn't fix this problem. Just re-install the damn thing. And if like me you experience the "SQL Server Backward Compatibility" error and you're running SP1, repair it by using "SQLServer2005_BC.msi" that gets extracted during the SP1 install. It's abit tricky since that file can only be found during the SP1 install - the installer gets rid of the install folder at the end of the installation.
• If you're having SharePoint issues and you're only using SharePoint for TFS (which was the case here) - same thing, just re-install and remember to choose "advnaced install" and configure your installation as a web farm. TFS 2008 supports SharePoint 3.0, so you might as well upgrade.
  

I hope this post will save somebody the pain I've just been through.