The Effect of CPU Caches and Memory Access Patterns

In this blog, I will provide you with some “Grade of the Steel” background information that will help you understand CPU caches and their effects better.

As we move towards a future where power consumption in the server room begins to make a big dent in the balance sheet – it becomes important for programmers to fully understand hardware and make the best possible use of it, instead of floating around in the leaky abstractions of the virtual systems. Even when we take the old enemy of I/O out of the equation, there are other bottlenecks we need to worry about: DRAM access time being one of the most important.

Table Pattern: Database Message QueuesTable Pattern: Database Message QueuesTable Pattern: Database Message QueuesTable Pattern: Database Message QueuesTable Pattern: Database Message Queues

2012-05-25
Databases
20 Comments
Latching, OLTP, Patterns, Performance, Relational

At the last SQL Bits X I held the FusionIO fireside chat during the launch party. During this presentation, I demonstrated how it is possible to build a table structure inside a relational engine that will act is a message queue and deliver nearly 100K messages/second.

Why You Need to Stop Worrying about UPDATE Statements

2012-04-27
Data Modeling
5 Comments
Data Warehouse, FusionIO, Keys, Patterns, Performance, Relational, SQL Server

There seems to be a myth perpetuated out there in the database community that UPDATE statements are somehow “bad” and should be avoided in data warehouses.

Let us have a look at the facts for a moment and weigh up if this myth has any merit.

Setting Yourself up for Windows Debugging

Lately, I have been doing a fair amount of debugging, and helped other people set up their debuggers.

It can be very painful to make this work. To assist up and coming debuggers, I put my notes together in this blog. Here is how to get things going.

When Database Statistics are not Enough – Search Patterns

2012-02-21
Data Modeling Databases
4 Comments
Histogram, I/O, Patterns, SQL Server, Statistics

Co-author: Lasse Nedergaard

Yesterday, Lasse ran into an issues with a query pattern in the large database that he is responsible for. Based on our conversation, we wrote up this blog and created a repro.

The troublesome query we were debugging executed like this:

Find a list of keys values to search for
Insert these keys in a temp table – lets call this the SearchFor table
Join the temp table to a large table (lets call it BigTable) and retrieve the full row from the large table

Why not use a correlated sub query in step 2? In this case, the customer in question had multiple code paths (including one accepting XML queries) that all needed to pass thousands of key to a final search procedure. They wanted a generic way to pass these key filters to to the final access of BigTable.

A DW Venn Diagram

Running Many Batch Statements in Parallel

2012-01-08
SQL Server Archive
7 Comments
Async, ETL, Parallelism, SQL, SQL Server, Threading

When designing highly scalable architectures for modern machines, you will often need to do some form of manual parallelism control. Managing this is not always easy, but in this blog I will give you one piece of my toolbox to help you.

We will walk through an example together, a tiny case study of a problem many of you will be familiar with.

Improving Slide Layouts– A Case Study

Very soon, it is conference time again, at SQLBits. As our team is preparing for this big event, I took a hard look at the slides we are using and the design we apply. While you may argue that spending time on slide design is a waste. I beg to differ. Using slides that are easily readable, beautiful and speak with clarity is important: No matter how great your content is, my experience shows that the audience will also judge you on the quality of the slide material you use. I will also assert that that most people are visual learners, so you owe it to your audience to use well designed slides. But even if you still disagree and if you don’t believe good slides help visual learners, consider this simple observation: if your slides look sloppy, it reflects on your character.

Don’t Become a One trick Architect

2011-12-08
Data Modeling
12 Comments
Architecture, Cursors, Data Warehouse, ETL, HADOOP, Hardware, OLTP, XML

We are near the dawn of a new workload: BigData. While some people say that “it is always darkest just before the dawn”. I beg to differ: I think it is darkest just before it goes pitch black. Have a cup of wakeup coffee, get your eyes adjusted to the new light, and to flying blind a bit, because the next couple of years are going to be really interesting.

In this post, I will be sharing my views on where we have been and a bit about where we are heading in the enterprise architecture space. I will say in advance that my opinions on BigData are just crystalizing, and it is most likely that I will be adjusting them and changing my mind.

Implementing MurmurHash and CRC for SQLCLR

2011-12-07
Programming
2 Comments
.NET, C#, Hashing, Performance, SQL, SQL Server

As we saw in my previous post, the build in hash functions of SQL Server were either expensive with good distribution, or cheap, but with poor distribution. As a breath of fresh air, let us look at a useful magic quadrant:

Previous Page «‹3 456 7 ›»Next Page