Michael J. Swart

April 20, 2009

Indexing Foreign Keys?

Filed under: SQL Scripts,Technical Articles — Michael J. Swart @ 12:11 pm

No, this article should be titled Indexing for Foreign Keys (or even Indexing for Forty Foreign Keys)

So your application is humming along and your database has been playing nicely with the app. But the database is getting a little large and it’s time to manage that. There are a lot of different ways to approach this common issue including short term solutions like:

  • Buying more disk space to host all this extra data.
  • Using compression to make the most of the disk space you’ve got.

And of course, there’s the archive/purge/restore approach.

At some point during the purge process, you may find yourself issuing DELETE statements. And it’s almost inevitable if you’re not doing something like dropping or truncating shards or partitions. When deleting rows from a table, in order to maintain data integrity SQL Server will always check foreign keys to make sure that no other row is referring to the row you’re deleting. This is, after all, the whole point of a foreign key and it’s the R in RDBMS. This check can cause performance problems for your delete statement if the database is not indexed properly. Such performance issues may have gone unnoticed if tables have rarely had any rows deleted in the past.

I wrote a query against some the DMVs available in 2005 and 2008 that can report on the tables that might cause scans when deleting rows. If the numbers are large enough, then you’re going to have trouble deleting many rows from these tables without adding an index (hence the title Indexing for Foreign Keys).

WITH my_foreign_key_list AS (
   SELECT 
       fk.name AS foreign_key, 
       fk.referenced_object_id,
       OBJECT_NAME(fk.referenced_object_idAS referenced_table,
       (
           SELECT CAST(c.name AS NVARCHAR(MAX)) + N',' AS [text()]
           FROM sys.foreign_key_columns AS fkc
           JOIN sys.columns c ON c.OBJECT_ID fkc.referenced_object_id
               AND c.column_id fkc.referenced_column_id
           WHERE fkc.parent_object_id fk.parent_object_id
               AND fk.OBJECT_ID fkc.constraint_object_id
           ORDER BY fkc.constraint_column_id
           FOR XML PATH('')
       ) AS referenced_columns,
       fk.parent_object_id,
       OBJECT_NAME(fk.parent_object_idAS parent_table,
       (
           SELECT CAST(c.name AS NVARCHAR(MAX)) + N',' AS [text()]
           FROM sys.foreign_key_columns AS fkc
           JOIN sys.columns c ON c.OBJECT_ID fkc.parent_object_id
               AND c.column_id fkc.parent_column_id
           WHERE fkc.parent_object_id fk.parent_object_id
               AND fk.OBJECT_ID fkc.constraint_object_id
           ORDER BY fkc.constraint_column_id
           FOR XML PATH('')
       ) AS parent_columns
   FROM sys.foreign_keys fk
),
my_index_list AS 
(
   SELECT t.OBJECT_IDt.name AS tablename,
       i.name AS indexname,
       (
           SELECT c.name N',' AS [text()]
           FROM sys.index_columns AS ic
           JOIN sys.columns c ON c.OBJECT_ID ic.OBJECT_ID 
               AND c.column_id ic.column_id
           WHERE ic.OBJECT_ID i.OBJECT_ID
               AND ic.index_id i.index_id
               AND ic.is_included_column 0
           ORDER BY ic.key_ordinal
           FOR XML PATH('')
       ) AS index_cols
   FROM sys.indexes AS i
   JOIN sys.tables t ON i.OBJECT_ID t.OBJECT_ID
   WHERE i.index_id 0
)
SELECT mfkl.referenced_table AS [When deleting a row from this table...],
   SUM(p.rowsAS [the number of rows that must be scanned is...]
FROM my_foreign_key_list mfkl
LEFT JOIN my_index_list missing_parent_indexes
   ON mfkl.parent_table missing_parent_indexes.tablename
   AND missing_parent_indexes.index_cols LIKE mfkl.parent_columns N'%' -- index covers the parent columns
JOIN sys.partitions p -- just to get rows
   ON p.OBJECT_ID mfkl.parent_object_id
   AND p.index_id IN (1,0)
WHERE missing_parent_indexes.tablename IS NULL -- with the LEFT JOIN, this is the 
                                               -- "missing" part of missing_parent_indexes
GROUP BY mfkl.referenced_table
ORDER BY SUM(p.rowsDESC

You’ll get results that give you a sense of what it’s going to cost to delete a row from a given table.

As an example from AdventureWorks, the query points to Sales.SpecialOfferProduct as the largest potential problem, and sure enough a delete on that table causes a scan on Sales.SalesOrderDetail

March 26, 2009

Using Database Snapshots as Transactions

Filed under: SQL Scripts,Technical Articles — Michael J. Swart @ 7:17 am

Linchi Shea recently asked whether people were using database snapshots or not. The consensus seems to be that they’re used during testing and rarely in production.

I use them for testing as well. They’re very good when used this way. Being able to revert to an snapshot is what makes the feature useful. It’s not that hard to understand really. It’s very very much like a database transaction.
Here’s the template that I use, just cut and paste into management studio and type Ctrl+Shift+M in order to replace the template values. It’s not quite foolproof, you do need to understand snapshots and edit the script to specify data file names. But maybe you’ll find it useful.
-- "begin tran"
   /*
       declare @cmd nvarchar(max)
       select @cmd = N'use <dbname,,> exec sys.sp_helpfile'
       exec sp_executesql @cmd
   */
CREATE DATABASE <dbname,,>_Snapshot ON
NAME datafileFILENAME 'C:\Temp\<dbname,,>_Snapshot.ss' )
AS SNAPSHOT OF <dbname,,>

-- "rollback"
   /*
   select spid from sys.sysprocesses where dbid = db_id('<dbname,,>') and spid > 50
   */
USE master
RESTORE DATABASE <dbname,,> FROM DATABASE_SNAPSHOT '<dbname,,>_Snapshot';
GO
DROP DATABASE <dbname,,>_Snapshot;
GO

-- "commit"
DROP DATABASE <dbname,,>_Snapshot

September 29, 2008

A better XML shredding example

Passing data into the database as a set has always been a challenge. There have been a number of approaches used for various purposes. And those approaches are discussed in many different places already.

If you only have to support SQL 2008, then table valued parameters are definitely the way to go.

If you have to support SQL 2005 (like myself) then other methods have to be used. Such as by parsing CSVs. Or my new favorite method of shredding xml.

Shredding XML
I like this method because it maintains data/script separation which is important from a security point of view.

One way of shredding xml is by using the nodes() method of the xml data type. The official documentation is here, but it wasn’t clear how to use this method for the business case I mentioned.

Here’s my example which I use as a template. Maybe you’ll find it useful too:

DECLARE @data XML
SET @data = '<root xmlns="http: //www.MySampleCompany.com">
 <book author="J K Rowling" title="Philosopher''s Stone">
  <chapter number="1" name="the boy who lived"/>
  <chapter number="2" name="the rest"/>
 </book>
</root>';

WITH XMLNAMESPACES ('http: //www.MySampleCompany.com' AS MY) 
SELECT 
   chapters.node.value('../@title', 'nvarchar(50)') AS bookTitle,
   chapters.node.value('../@author', 'nvarchar(50)') AS bookAuthor,
   chapters.node.value('@number', 'int') AS chapterNumber,
   chapters.node.value('@name', 'nvarchar(50)') AS chapterName
FROM @data.nodes('//MY:chapter') AS chapters(node)

The results look like this

bookTitle             bookAuthor  chapterNumber chapterName 
--------------------- ----------- ------------- ------------------ 
Philosopher's Stone   J K Rowling 1             the boy who lived 
Philosopher's Stone   J K Rowling 2             the rest

Also, if you don’t use namespaces with your XML, you just omit the WITH clause.

Update!
Check out this post to see how this feature looks from the app side (using c#).

April 30, 2008

What are you waiting for?

Filed under: SQL Scripts,Technical Articles — Tags: , , — Michael J. Swart @ 5:04 am

Update July 5, 2011: Many people are coming to this blog post through search engines trying to figure out what LAZYWRITER_SLEEP means. I looked at this article and determined that it’s not clear. I call the wait type ignorable, but I don’t say why. So here’s why: LazyWriter_Sleep:  Happens when a lazy writer task is suspended and waits to do more work. Since lazy writer tasks are background tasks, you don’t have to consider this state when you are looking for bottlenecks. Hope this helps… By the way, a much better article than this one is here: Wait Statistics by Paul Randal. What follows now is the original article:

The sys.dm_os_wait_stats helps to see where server processes are spending most of their time waiting. If the top cause is LAZYWRITER_SLEEP or SQLTRACE_BUFFER_FLUSH, then you’re all right. If not, then you’ve got some digging to do. Maybe start with sys.dm_exec_requests.

What are you waiting for now?

The times that are shown are cumulative since the start of the SQL Server service. I use this script to figure out what SQL Server processes are waiting for now:

select * into #BufferWaitStats from sys.dm_os_wait_stats

-- wait ten seconds or so
select ws.wait_type,
      ws.waiting_tasks_count - bws.waiting_tasks_count as waiting_tasks_count,
      ws.wait_time_ms - bws.wait_time_ms as wait_time_ms,
      ws.max_wait_time_ms,
      ws.signal_wait_time_ms - bws.signal_wait_time_ms as signal_wait_time_ms
from sys.dm_os_wait_stats ws
join #BufferWaitStats bws
      on ws.wait_type = bws.wait_type
order by wait_time_ms desc

--clean up
drop table #BufferWaitStats
« Newer Posts

Powered by WordPress