EMC Live! WebCast Series – SharePoint

March 4, 2010

Hi Folks

Back again – been a while, aye but have been working on some exciting stuff, which I will share with you in due course.

So today if anyone has an hour to kill, Eyal and I are presenting an EMC webcast (part 1 of a 4-part series) on SharePoint and touching on what’s coming in SharePoint 2010.



Today will be

Thursday, March 4, 2010 – 12 pm PT / 3 pm ET
Learn how to design your SharePoint infrastructure to ensure optimal performance and scalability, as well as leverage the benefits of virtualization.

Part 2 is Dave from our SourceOne engineering wing

Thursday, March 11, 2010 – 12 pm PT / 3 pm ET
Find out how to mitigate risk, reduce costs, and improve SharePoint performance with EMC SourceOne.

Part 3 is Eyal and I again

Thursday, March 18, 2010 – 12 pm PT / 3 pm ET
Learn how to design, deploy, and manage your SharePoint infrastructure to ensure availability and rapid recovery, as well as understand what options are available—from native SQL Server functionality to array-based replication.

Part 4 is another Dave from another engineering wing, EMC Documentum
Enhancing SharePoint to Meet Your Information Management Needs

Thursday, March 25, 2010 – 12 pm PT / 3 pm ET
Discover how EMC Documentum integrates SharePoint into your broader information infrastructure, enabling you to cut operational costs and reign in server sprawl.

Hope to see ye there !

But I will share the content with ye later anyways 🙂



Poll: What SQL recovery model do YOU use for your SharePoint databases?

November 16, 2009

I’d be very interested to know what kind of SQL recovery model do YOU use with your SharePoint databases.

The poll is anonymous, so feel free to be honest!

If you use multiple recovery models, specific to the SharePoint database type, click both options.



SharePoint and SQL Databases

November 15, 2009

Hey folks,

This week I am at presenting SharePoint and Hyper-V information to both EMC and Microsoft personnel @ the Microsoft campus, Building 33 in Redmond, WA.  Strangely, its not raining! 
I will be covering topics, such as best practices, Hyper-V virtualization, backup and recovery and DR.  I hope to share these presentations with you once the conference is over, so stay tuned.

On to the real topic…SharePoint & the proliferation of SQL databases.

SharePoint’s main stay of information is in the form of SQL databases. 

In a typical SharePoint SQL Server I would categorize these databases in the following four layers;

===SQL System Databases   (created when SQL is installed)
           -Master, Model, MSDB, TempDB

===SharePoint configuration Databases   (created when SharePoint is installed)
           -SP_Config, etc

===SharePoint content databases   (created at the end of SP install, portal & content creation)
           -WSS_Content_* (Portal), 
            SharePoint_AdminContent_* (Central Admin)
           User-defined content databases (eg ContentDB01, 02, 03)

===SharePoint Shared Services Provider databases   (created with SSP & application configuration)
           -SharedServices_DB  (SSP Configuration database)
           Shared Services applications 
                 -SharedServices_Search_DB   (actual Office Search “Osearch” database)
                 -WSS_Search_{hostname}       (WSS SPsearch DB – per host)

You need to follow standard SQL best practices, including storage BPs to ensure good SharePoint performance, granular backup and recovery and efficient disaster recovery.  Agreed.
(I will go into more best practices for SharePoint SQL storage in a seperate post, let’s stay with this for now)


SharePoint does not allow the user to specify where the SQL database data and logs files should reside, and so the default database locations will be used.  The default databases data and log files locations are a part of the SQL instance configuration.

These are recorded in the registry, specific to the SQL instant, e.g.
             \DefaultData               (Default database data file location)
             \DefaultLog                (Default database log file location)

Easier is to use SQL Management Studio
   –  Right click the SQL instance, Properties, Database Settings -> Database default Locations.

Today, we have two choices in ensuring that SharePoint SQL databases are in the right locations…

1) Change the default SQL data file location prior to the SharePoint configuration task (e.g. create a SSP)
2) After the SharePoint configuration task, bring down SharePoint hosts, detach, relocate and re-attach the SQL databases.

I perfer #1 myself 🙂

So, here are the recommended sequence of steps to take

Recommended sequence

1) Install SQL with advanced options
  -ensure that master, model, msdb locations are correctly set.  
  -ensure that tempdb is on different LUNs, ideally filegroup the datafiles

2) Change the default database file locations to your SP Configurations volume
          -then install SharePoint.

3) Change the default database file locations to your “basic content” (or SP Configurations) volume
         -then create your SharePoint portal(s).

4) Change the default database file locations to your SSP & Search Configurations volume
         -then create your SharePoint SSP and add SPsearch roles to hosts

5) Change the default database file locations to your SSP Search Database volumes
         -then create your SharePoint SSP Search application and associate a content source


Now, user-level content databases is trickier…
You dont want to have to follow this procedure every time as many content databases will be created in time. 

My recommendation would be;

1) logged in as the SharePoint system account in SQL, manually create content databases in the right storage locations
2) then use either Central Admin or STSADM to attach an existing SQL Content Database

  a) Central Admin way 
    – Central Administration > Application Management > Content Databases
       – specify the name of the existing SQL content database


     b) Stsadm way – stsadm -o addcontentdb -url (URL) -databasename (ContentDB name) -databaseserver (SQL name)

  stsadm -o addcontentdb -url http://portal.sps.com/site01 -databasename ContentDB01 -databaseserver SQL1

You should not need to specify the username/password as you will use a trusted connection within your domain.

I am putting in an enhancement request to Microsoft to allow SharePoint admins to specific the directory locations from Central Admin/STSADM/Powershell in future.

Every so often, especially with dispersed power users (capable of creating content databases), full audits of SharePoint database files should be carried out.  It is vital to ensure that all databases are protected.

Hope this helps people

Fancy having fully-automated site Disaster Recovery?

November 4, 2009

A critical application…
…nearly as much $$$ spent on your DR site as your Production site…
…hours of time spent on defining & refining your disaster recovery procedure…
…never mind the training & updates from testing…
…varying bandwidth between sites, sometimes not enough…
…want to be able to recovery your SQL server back to 10:22:16…
…and that was 2 days ago…
…need that data back within minutes…

Well you can…

EMC RecoverPoint, EMC’s CDP (Continous Data Protection) product provides near instantanous roll-back capability.

 The majority of Operating Systems and applications are supported.  Data reduction* and WAN bandwidth compression – native.  Non EMC storage arrays – supported.  Long distances – how about 2000km? – supported.

Combine RecoverPoint and EMC Cluster Enabler together (called RP/CE for short) and the solution gives you just this.

RP/CE is what is called a geographically dispersed clustering solution, but alot more…
…RP/CE allows one or many clustered applications or Hyper-V VMs to be failed over in a minimal amount of time to a disaster recovery cluster node(s)   – in my Proven Solution – try 3 production / 3 DR nodes. 

I could even run some VMs on the DR cluster nodes (if your network supports this of course) and some on Production.  RecoverPoint/CE doesn’t mind, it supports bi-directional replication. 
EMC understands nowadays DR sites are too costly to just leave idle. 
RecoverPoint natively supports bi-directional replication in the same RecoverPoint installation.

The slick part of RecoverPoint/CE is that once operational, all the user needs to know is how to use normal Microsoft clustering and cluster administrator console.  RP/CE adheres to all Microsoft failover clustering requirements.  It is installed as a clustered resource and is added to each cluster group which needs RP/CE’s protection.

Say, you have a 2-node (1 active / 1 passive) Hyper-V server running your virtualized SharePoint farm.  Down goes your production site.  Within a few minutes, all your SharePoint virtual machines are up and running on the DR side again, with the latest image of your data.  It’s that simple.

I was at the EMC booth as SharePoint Conference 2009 ( SPC09 ) in Las Vegas in October showcasing this and I must say…visitors were very impressed…most especially the folk who have been through the pain of recent DR planning..

* as a classic example, in something like an OLTP environment, if an 8k SQL data page is changed by say 200 bytes, the entire 64k block is written back down to filesystem.  Without RecoverPoint’s data reduction, all 64k of data is shipped across the wire to the DR site.  With data reduction, only the 200 bytes plus some checksum data is sent across the wire – AND compressed! clever.

For a recorded demonstration of the Proven Solution I am working on please see below;

In this demo, I would like to show you the power of EMC RecoverPoint and Cluster Enabler (RP/CE) in providing fully automated Disaster Recovery in your environment. In this use case, a busy enterprise SharePoint farm hosting 240,000+ busy users will expect a full site disaster and RP/CE automates disaster recovery of the farm in minutes.

The environment consists of a 6-node Hyper-V cluster, (3 Active/3 Passive) using iSCSI connectivity to an EMC CLARiiON CX4-240 storage array

I will share more information on this solution as it evolves.

Some more doc resources 

EMC RecoverPoint/Cluster Enabler – a detailed review
Disaster Recovery for Windows using EMC RecoverPoint/CE
EMC RecoverPoint/SE for CLARiiON Cx4
DR in a geographically dispersed cross-site virtual environment

I would love to get some feedback on what people think of something like this, not necessarily this EMC solution, but geo-clustering in general…
If anyone would like some serious detail into how RecoverPoint/CE works, I can gladly provide same as a blog post.

Thanks, James.

EMC Replication Manager for SharePoint – watch this!!!

October 16, 2009

As I mentioned before about a special project I was working, well this is it.  SharePoint DBAs SHOULD be excited.

*configuring backup protection for the whole SharePoint farm in less than 5 minutes!

*full backup of an active (240,000 heavy users) SharePoint farm 
   – 3 hours 11 minutes. online, no disruption. 
   – 1.5TB of user content, 2.5TB of SharePoint files.

* incremental backup (with a daily change rate of 1%), 
   – 11 minutes

* restore a 100GB content database 
     – 7 minutes

* perform item-level recovery from a backup in minutes
   – without distruption, without a recovery farm

This blog is all about making life easier for the SharePoint Admin, users and architects.  This product brings that thought much closer to reality.  EMC Replication Manager for SharePoint 5.2 SP2.

A single application, central console, simple, easy. A storage guy, a DBA, a windows guy – they can all relate and understand it….

While the Blueprint documentation has not yet hit EMC.com, I share it here with you now.


For more information to what EMC can offer on SharePoint go to http://tinyurl.com/EMCMOSS
Here is a diagram of the production farm.


To illustrate the point, I have created 4 video demonstrations;

1) Creating that application protection (backup configuration & scheduling)

2) Running a backup against a very busy SharePoint farm (worse case scenario test)

3) Restoring a content database from a single user interface

4) Using the combination of Kroll Ontrack Powercontrols and EMC Replication Manager to simplify item-level recovery

So, eh, what did you say you do again?

October 5, 2009

To follow on from where I left off….let’s face it, it was getting soppy!

What exactly do I ? 
  I design and build solutions around SharePoint (and SQL).  I illustrate how a customer’s life can become easier through technology, but hold on, through a well tought-out and well tested solution, not just a badly packed kebab of products and hardware…

The primary output of my work is documentation and consulting to EMC personnel (field, engineering, support, etc)
A picture is worth a thousand words…

A quick overview of what I have been involved in…

May 2008
-Enterprise SharePoint in a VMWare 3.5 environment on our mid-range storage platform the EMC CLARiiON CX3 (nowadays is CX4).
  Take a wee peek at the Reference Architecture  (short overview document of the solution).
  If you would like the hefty Integration Guide, let me know, people call it the SharePoint infrastructure bible.

We tried to make the solution suffer.  Honestly.  We went full misfit.
We pulled power from 1 of the 3 ESX servers in the HA cluster, we pulled CX3- storage processors, we pulled disks from R5 raid groups, we profiled the effects of VMotion on a Web Front End, we profiled when DRS is configured too aggressively, we failed over the physical SQL cluster backend during full-flight load…we make pretty graphs and tell the true stories.  This is what we do.

After that I decided to make best use of a another case, exactly identical to my virtualized solution but a physical environment and compare and contrast savings that virtualization can make to customers in real terms.  The magic number 74%!  Savings.  Chad Sakac, EMC’s VMWare technology alliance guru, used it, it got take up, and was presented at VMWorld – point here – not me – the tests – these tests are invaluable in getting the “real” data out there.  EMC invests ALOT of money to allow us to do this in the labs, trust me.

Being the majority shareholders of VMWare, we in solutions decided not fall to that.  As soon as Hyper-V was in a steady state near beta end, we decided to drop VMWare as our hypervisor in my workstream for virtualized SharePoint on CLARiiON and test out what Hyper-V meant to us and customers.

March 2009
We completed our testing on that solution.  Reference Architecture
Interesting, in this use case we decided to try to lower the cost of the entire solution by using iSCSI where we could (why not, the array supported both FC and iSCSI).
We tried out SCVMM (System Center Operations Manager), in conjunction with SCOM (Systems Center Operations Manager) in order to see how PRO (Performance Resource Optimization) worked.  PRO is the equilivent to VMWare DRS.  We did some really interesting tests with some really interesting results.  I’ll share the Integration Guide with you for a story 🙂

May and June 2009
EMC’s flagship storage array, the Symmetrix took a fair leap into the next decade of technology with the VMAX offering.
We took a look at the array, and said to ourselves, “what would a customer do”.  Buy it JUST for one application, no, buy it for maybe the top three mainstream applications, well yes.

So, we set off from the Shire, three application experts in SQL, SharePoint and Exchange, a storage nut and a VMWare nerd and decided on what we would do.  Virtualization – eh, yes…..maximum efficiency for storage – yes (Virtual Provisioning)……storage tiering for performance – yes (Virtual LUN).
Protect the entire environment online with no production impact, well, let’s see….

The Reference Architecture, the White Paper

July 2009
I started to get excited. 
I know what difficulties SharePoint admins have in their daily lives for enterprise-level SharePoint backup and recovery, not to mention the ability to recovery subsets (like single documents) from these backups.  Recovery farm requirements, etc, its just messy.

I knew what EMC had planned in September….. wait for my next post, where I will reveal…this is for you SharePoint admin!


Good Day to you..

June 30, 2009

Hello there.

Before I start writing ….

I want to set the stage here…
 I didn’t think I liked blogs, I actually thought they were a bit self-indulging!
   I certainly STILL know I dont like writing about myself!
     But, every day I now see how blogs _help_ people like you and me.  
       It is to me, a great way for people to distil information in a friendly format.

The reason for this blog is you

I feel like I am in a privileged position in what I do as a profession and I want to be able to share my experiences and information in order help you in your daily endeavours.  I had been asked before by people if I had a blog, why I didn’t and would I consider writing one.  I resisted, in hindsight, wrongly.  I said to myself, the next person who asks me, that will be the trigger-pull I need to start…..that was today…

To state clearly, I am not out to promote myself, some of my blog posts might be total hogwash or is not what you see in your environment (I want to hear about that), and some will indecently try to promote some of the technology my company has to offer.

My name is James Baldwin and I work in EMC Corporation, in Cork, Ireland.  I am  (wait for this title!) the EMC’s Global Solutions SQL and SharePoint Lead Engineer.  Waiting on the business cards, it will be a riot 🙂

As you can guess, I don’t take myself seriously, but I DO take what I do very seriously.  I suppose people call me, amoungst other things, an EMC and application evangelist, I actually perfer to call myself a customer evangelist.  The former falls into place.  That is far more important in my eyes, and I hope that reflects in my subsequent blog posts.

I started working in EMC in 2001, when the shares were still soaring and business class travel on flights was standard. 

Before that, I came from 3 years in a special wing of Dell engineering where we built custom or complex desktop and server builds.  All OS’es, all hardware, engineered the first Redhat 6.0.x orderable on PE server, drowned in OS/2 Warp for a very special customer for a bit, and importantly delved into all kinds of challenges which customers had.

If you really want to rewind further, I did a Bart Simpson on my dad to force him to buy me a Spectrum 48K at the age of 13.  Now he has his own back on me any time he has an “anomoly” with his home PC.  IT Karma.  I remember loading VMWare on Slackware 4.0, seeing my own PC booting inside itself, displaying a gammy pseudo Pheonix BIOS et al, saying to myself “Jeez that’ll never catch on!” 🙂  Think of the shares….Think of the shares….Forget the Sports Almanac, if I get a working DeLorean, I want the IT Almanac to go to 1996 with.  Enough of that…

I arrived into EMC in a technical support capacity, supporting their enterprise backup product at the time, EMC Data Manager (EDM), which ran on Solaris, slightly different beast Linux, but a great OS I must say for multi-threaded applications with some really well thought-out debugging tools.  We backed up everything, all mainstream applications, all mainstream OS’es.  I quickly understood that regardless of the severity of a call, absolutely nothing is trivial to a customer.

It may well be trivial to someone preaching the topic, but when you are the customer, responsible for a live user environment where a critical business application depends on you and your team member, it’s a whole lot more serious.  Go on, see if you can crack a joke with a customer who called you looking for help and guidance because their SAP instance is down and need to recover ASAP.  My record for affected users, 325,000.  Wont say why, who or how, but we got it fixed and afterwards figured out what went wrong, why and how to prevent it.  I must say, actually, I can easily say quality and customer focus was driven into us in technical support.  While the job was sometimes stressful, I loved every minute of that job due to the amount of satisfaction in helping solve problems for people.  That and the fact the next support call was like a box of chocolates…yes, the guy on the bench…

I remember having a customer call me directly and say “James, dont laugh, I just blicked SG2 on Exch-04”.  1,400 users.  These things happen every single day.

Along came technology, disk costs lowered, and this funky thing of point in time replication became a household name, well ok in the storage nerd’s house.
EMC Replication Manager came along and changed things for us in support. 

I changed role slightly and had more of a free hand in making things better for the support team in documentation, training and mentoring.  In this role, I now understood more.  I understood the customer’s problems, but as importantly, I understood the challenges of my fellow technical support people in dealing with such events, in gaining experience and knowledge and being able to apply it.  Just as we stabilized our perceptions on Replication Manager and these Virtual Tape Library (VTL) Units, along came RecoverPoint!   Again, another significant step in technology.  Time to write more “uncovered” documentation to help ourselves figure this stuff out and to chase engineering groups to make their products more sustainable.

I started to understand how technology could actually help us (you and me) in our daily grind.  Those poor customers who lost data once had to recovery from tape, could then recently recover from a point in time, could now recover to any point in time.  Ignore vendors, the story is there.  Technology was getting better.  In some circumstances, it was really helping, ie recovery.  But, technology was not helping in many respects.  Added complexity, the same old human traits (mistakes), mis-placement of technology, and so on.  Its there today and will be there tomorrow.

I began to travel to customer sites to assist on the ground with highly technical issues, nothing in a specific area, but more around what environment or “solution” the customer was operating in.  I now had the four angles.  The customer, the technical support person, the product, and the person who designed the environment (be that person a customer, fellow employee or a third party consultant).

I thought I had a pretty good idea and more importantly, an appreciation of what problems exist in the IT industry…now how could I apply my knowledge more effectively, to prevent those customer issues in the first place?…

Wind forward to today…
I joined the EMC Global Solutions Center in 2007, after 6 good years in support.  When I saw the job posting and read what it was about, I said to myself “this is for me”.  The position was for a SharePoint solutions engineer.

The Global Solutions Centers (6 of them scattered around the world) are engineering centers of people who take a given solution, design it, built it, test it, break it, analyze it and document it.  In a nutshell.  We hope to find bugs and when we do we work them out with the product owners (EMC, VMWare, Microsoft, Oracle, etc).  We test the environment to scale.  We document the best practices.  We document what not to do, and why not.  We try to take the guess work out of a solution for a customer.

“What do a need to run 240,000 heavy SharePoint users in a tiered environment on Hyper-V with rapid backup” – we answer these questions.

To apply that in context…
We work with customers, all EMC practice and engineering groups and application vendors (e.g. Microsoft, Oracle, SAP, etc) in trying to understand what the most common use cases are for certain applications and environments.  Through customer and field feedback, we then decide on what demands and requirements customers have on such an environment. 

My team is focused on SQL and SharePoint solutions, but dont let your questioning stop there.  Half the battle is knowing I am in a good team of multi-disciplined people, I’ll get the answer.

We go ahead and built out the racks, servers, network & fibre channel switches, storage etc in our customer integration labs.  “For this project, I’ll have a 6x 24-core servers please :-)”
Using industry-standard load generation tools, we performance iterative tests on a good datacenter day and on a bad armageddon datacenter day, functional testing, core switches dying, Clustered physical or virtual servers being unplugged (oops!), half the storage array going down – would never happen 🙂  You get the picture.

We test and profile the bad things, so you as a customer dont have to guess what if….

Remember, we test with two things in the forefront of our mind at all times – application and customer.
As a customer, I dont really care that an EMC switch, disk, cable or software component has died, I just care about my app.

We take that approach.  If we see something we don’t like, we let the relevent people know “You’ve got to fix this, don’t let a customer suffer this” is the message.
We are the “external” customer that product groups always wanted, but they dont really know it. 

I will share our experiences with you.  I will share “look what we found” with you in concise, technical details.  I want you to share your experiences with me.  The more I know about your struggles, the more I hopefully will be able to help.  I am entirely willing to test something you hit in our labs if we have the given time and capacity.


In summary, I’m here to help…

Thank you