SharePoint Notes

Bleeding on the cutting edge …

Archive for the ‘Capacity Planning’ Category

SharePoint and very large databases

Posted by Christian Dam on October 27, 2009

I got a very interesting call from a colleague the other day. He had a customer with content database which apparently was approaching 1TB! My colleague of course told the customer that it was not aligned with best practices and that it could cause problems down the road to which the customer replied with a smile: “Well, if that happens we just throw additional resources at the SQL Server. Thanks for playing, now leave me alone”

So, he called me:  “Does it sounds plausible? SQL Server 2005/2008 can support exabytes of data so what’s the problem?”

Well, I went to work and starting digging up some information. My main bibles SharePoint 2007 Best Practices and Inside SharePoint 2007 Administration both referred to Microsoft guidelines which recommend to limit the content databases to 100GB:

Limit content database size to enhance manageability

Plan for database sizing that will enhance manageability and performance of your environment.

  • In most circumstances, to enhance the performance of Office SharePoint Server 2007, we discourage the use of content databases larger than 100 GB. If your design requires a database larger than 100 GB, follow the guidance below: 
    • Use a single site collection for the data. 
    • Use a differential backup solution, such as SQL Server 2005 or Microsoft System Center Data Protection Manager, rather than the built-in backup and recovery tools. 
    • Test the server that is running SQL Server 2005 and the I/O subsystem before moving to a solution that depends on a 100-GB content database. 
  • Whenever possible, we strongly advise that you split content from a site collection that is approaching 100 GB into a new site collection in a separate content database to avoid performance or manageability issues. 
  • Limit content databases that contain multiple site collections to approximately 100 GB.

However, the recommendations didn’t say anything about why 100GB is the magic number. MVP Ben Curry even at one point said “100GB doesn’t seem very big anymore 🙂 If you need larger site collections, optimize SQL and rock on… “ 7-8 month later, Ben had thought it over once again and had come to another conclusion: “So, use your head when architecting databases/site collections and don’t smoke crack. I’ve even said you can have monster content databases in the past – I was wrong. The only way I would now architect large content databases would be for fairly static data that did not have a large collaborative user population.”

OK, so why is Ben advising us not to smoke crack? In other words, why are large content databases evil?

Obviously large databases makes it difficult to administer the databases and to maintain a realistic back/recovery plan due to the sheer amount of data:

  • backup and restore take a long time (normally 150GB can be backed up in a 4 hour window)
  • the SLA may be in jeopardy
  • it is difficult to periodically test and verify the backup/restore procedure
  • database attach is slow 
  • upgrading WSS/MOSS service packs will be slow
  • I am sure some custom code start to suffer as well

Note: the backup/restore timing issues normally doesn’t apply to those customers using SQL Mirroring or SQL log shipping.

Additionally – and probably at least as important: more data will lead to more SQL I/O which again lead to more locking and blocking on the SQL Server backend. All data in a site collection share the same table and if that table is blocked, access to all sites in the site collection is blocked! It is a normal SQL Server thing and as such it is not even visible in the event or trace logs.

Splitting up the content database can help overcome most of these problems and it is fortunately a relative simple thing to do. Here’s how to do it on WSS and MOSS.

Advertisements

Posted in Capacity Planning, MOSS, WSS | 1 Comment »

Announcing the SharePoint Capacity Planning Tool – Solution Accelerator

Posted by Christian Dam on February 5, 2008

You are receiving this because you have signed up for the SharePoint Capacity Planning Tool Beta.

The Microsoft Solution Accelerator Communications and Collaborations group is pleased to announce the availability of the SharePoint Capacity Planning Tool for download on TechNet.

Are you planning a SharePoint installation for a customer? Chances are, you have a lot of questions.

  • What is the necessary hardware investment?
  • What kind of topology is needed to meet organizational requirements for availability and performance?
  • How will additional users from a recent merger affect deployment?

When planning a new SharePoint deployment, you will need a way to quickly characterize the general topology and minimum hardware requirements to support expected usage loads with adequate performance. You will need a way to explore plausible scenarios and get pointed in the right direction. The SharePoint Capacity Planning Tool gives you a quick way to draft topology and equipment sizing for your particular scenario and usage profile.

This solution reduces “SharePoint sizing” estimation efforts from a matter of days to a matter of hours.

Description of the Tool

Imagine a tool that you could use to input a few key facts about users, user needs, business requirements, and organizational resources in order to:

  • Build a topology model based on this input
  • Run simulations that show the impact of this topology
  • Generate a summary report that will help you evaluate alternatives

The SharePoint Capacity Planning Tool consists of System Center Capacity Planner 2007 (SCCP) models for:

  • Microsoft Office SharePoint Server 2007 (MOSS)
  • Windows SharePoint Services 3.0 (WSS)

The tool can be used for planning and feasibility studies of a deployment project to give you a rough estimate of hardware requirements. The tool can generate estimated performance metrics based on a simulation and provide useful Visio schematics of the proposed deployment to be included in documents and proposals.

You can obtain the SharePoint Capacity Planning Tool on Microsoft Download. For a short introduction to the tool and its usage, please go to this TechNet overview page: http://go.microsoft.com/fwlink/?LinkID=93030

Note: This Solution Accelerator needs SCCP 2007 to be installed first.

Downloads

SharePoint Capacity Planning Tool on Microsoft Download Center:
http://go.microsoft.com/fwlink/?LinkID=93029

System Center Capacity Planner:
http://www.microsoft.com/systemcenter/sccp/default.mspx

Regards
The Capacity Planning Tool Group

Posted in Capacity Planning, MOSS, System Center, WSS | Comments Off on Announcing the SharePoint Capacity Planning Tool – Solution Accelerator

System Center Capacity Planner 2007 RTMs

Posted by Christian Dam on February 4, 2008

You are recieving this email as part of your membership in the beta program for System Center Capacity Planner 2007.

System Center Capacity Planner 2007 RTMs

The product team is pleased to announce the release of Microsoft System Center Capacity Planner (SCCP) 2007! SCCP 2007 is a pre-deployment capacity planning and post-deployment change analysis solution for Microsoft Server products including Microsoft Exchange Server 2007. It employs state of the art modeling technology to bring unprecedented functionality and flexibility to the process of performance analysis and planning of distributed application deployments. It provides Information Technology professionals with the tools and guidance to deploy efficiently, while planning for the future by allowing for “what-if” analyses.

The new version adds the following new capabilities:

  • Application model for Exchange Server 2007
  • Extensibility with new models from Microsoft, including Microsoft Office SharePoint Server 2007 and Microsoft Windows SharePoint Services 3.0. A model for System Center Operations Manager 2007 will be available soon.
  • New, more powerful deployment wizards deliver unprecedented ease of use
  • Support for 64-bit processor architectures
  • Comprehensive reporting of disk IO and storage utilizations
  • Increased flexibility in customizing server usage profiles

Unlike the previous version, SCCP 2007 doesn’t require a Microsoft TechNet Plus or MSDN Premium subscription, and is available for anyone to download now on the Microsoft Download CenterPlease visit our website to learn more about SCCP 2007.

When will Capacity Models be available?

  • Exchange 2007 Model: This model ships with SCCP 2007 and is available NOW and included with the current download on the Microsoft Download Center.
  • Operations Manager 2007: This model is not yet available and the beta is closed. The final model is expected to ship in the 2nd quarter of calendar year 2008. Customers seeking capacity guidance should use the OpsMgr 2007 Performance and Scale Guide now available on the Download Center.
  • SharePoint 2007 Model: This model is not yet available but will be posted to TechNet shortly.

NOTE: If you already have installed SCCP 2007 beta, you have to uninstall it first before installing RTM.

Posted in Capacity Planning, MOSS, System Center, WSS | Comments Off on System Center Capacity Planner 2007 RTMs

System Center Capacity Planner – now with support for MOSS/WSS

Posted by Christian Dam on January 4, 2008

As you may already have heard, SCCP has been expanded with support for MOSS and WSS. This is really exiting as it provides a great, intuative and simple of try out you MOSS planning efforts in a simulated environment – just as it has been the case with Exchange for som time now.
The SCCP tool it selv is released as a RC and the MOSS/WSS models in public beta.
As a small example, I’d analyze a scenario where we deploy a single intranet farm for a company with 2170 employees with a HQ and three branch offices.
The employees are located as follows:
  • HQ: 2000
  • Århus: 100
  • Odense: 50
  • Sønderborg: 20

Step 1: Select the Capacity Model

Select Microsoft Office SharePoint Server 2007 (Beta Model) and Create a new Capacity Model

Step 2: Create the Farm

Select Add Sharepoint (MOSS) Farm and enter this information:

  • Farm Name: HQ
  • Local Client Count: 2000
  • SharePoint Deployment: Intranet
  • Usage Profile: Heavy Collaboration

Click OK and Next

Step 3: Add branch offices

Select Add Branch Office Profile and create the three branch offices:

Name
Local Client Count
Usage Profile
Århus
100
Heavy Collaboration
Odense
50
Average Collaboration
Sønderborg
20
Light Collaboration

Continue with Odense and Sønderborg. Click OK and Next when finished.

Step 4: Network Information

Select Specify Connectivity for each Branch Office and enter the information below and click Next.

Yeah, I know the bandwich is not enough, but we’ll correct that later 🙂

Step 5: Select Hardware

Select appropriate CPU and disk configurations and click Next

Step 6: Application settings

Select the High Availability and SQL Server Storage settings specified below and clik Next

Step 7: Model Summary

Based on out initial input, the Capacity Planner have now calculated what an optimal configuration might look like, including MOSS servers, roles and disk layout.

Click Finish

Step 8: Simulations!

Tada! We have now entered our initial questimate into the model and we are now ready to put it to the test! Select the Run simulation option to start the simulation

The simulation appears to have identified a couple of issues:

  1. The response times are slow, especially from the Sønderborg office
  2. The storage space utilization is high
  3. The CPU utilization is high (although not directly identified by the model)

Improvement 1: Storage space utilization

To fix the storage space issue, select Hardware Editor and then Device configurations. Choose the Disk Array view and navigate to the HQ\SAN Array

Change the Disk Count from 20 to 24 for both Volume 2 and Volume 4. Finally Save Hardware Library (Ctrl + L).

Rerun the simulation from the Simulation Results page to verify the effect of the changes.

The storage space issue has been resolved.

Improvement 2: CPU utilization

To address the CPU utilization, some bigger CPU’s are assigned to the SQL Servers and the Web Front Ends. This should also have a positive impact on the response times.

Hardware changes are done from the Site Topology page in the Model Editor. Right-click on SQL Server Cluster (Primary) and select Edit server hardware configuration.

From the Apply New Configuration drop-down box, select the 4 x 3.00 GHx Xeon-configuration and click OK. Ignore the excessive use of local disk in this configuration. Repeat this step for SQL Server Cluster (Failover).

Use the same method to add additional CPUs to the two Web Front Ends.

Finally, rerun the similation to verify the effect.

As expected it did wonders for the CPU utilization, but it had only a minor effect on the slow responce times. Our last improvement should take care of that.

Improvement 3: Slow response times

Since we just beef’ed up the SQL and web Front end servers, the slow responce times are probably due to slow network links. No big surprise, but sometimes it is easier to upgrade hardware than to upgrade physical network links.

To upgrade the network links, navigate to the Global Topology page in the Model Editor. Right-click on the HQ – Sønderborg link and select Edit connection details.

In the Connection Speed section, upgrade both Uplink and Downlink speed to 45 Mbps (T3) and click OK.

Repeat for the HQ – Odense and HQ – Århus links. The rerun the simulation to see the results

Great news! All issues are resolved and we should have a working configuration. Navigate to the Model Summary page in the Model Editor to see the final configuration.

Conclusion:

The SCCP looks very promising and will be a great tool in performance and capacity estimates. The way it is possible to change the model is fantastic and it offers much much more than we have exploided here. For example it is also possible to place servers in the branch offices to enhance peformance even more.

One big problem, I think, is that the hardware requirements and networks link speed in branch office scenarios seems to be way to high to reach the defined thresholds for a healthy system … or is it just the a problem with the thresholds?

Oh, well – the MOSS/WSS models are still only in beta and they may be tuned a bit going towards RC and RTM

Posted in Capacity Planning, MOSS, System Center | 3 Comments »