This is something that I get asked a lot, so here's an explanation by way of example:

I've got a motor driver board controlled by a Luminary LM3S610 controller. It has an H-bridge using PWM to control a DC motor driving a gearbox/wheel.
I can send commands to the device via a serial cable, and for the assorted commands, I'm having trouble debugging/validating that everything is doing what I think it does (Is that '+' command incrementing the speed correctly, etc...).

I've got a linux box as my host machine (but the instructions are basically the same for windows) connected to the device for programming via an ARM-USB-OCD from Olimex, although any openOCD supported JTAG device would work-- wiggler, whatever.

For programming, I usually just issue the command "make program" and make automagically issues the following command:

openocd -f cortex_ft2232_flash.cfg

The version of openocd I'm using is whatever I pulled from the berlios svn repository...

URL: http://svn.berlios.de/svnroot/repos/openocd/branches/cortex-m3
Repository Root: http://svn.berlios.de/svnroot/repos/openocd
Repository UUID: b42882b7-edfa-0310-969c-e2dbd0fdcd60
Revision: 285
Node Kind: directory
Schedule: normal
Last Changed Author: mlu
Last Changed Rev: 125
Last Changed Date: 2007-01-22 15:52:50 -0500 (Mon, 22 Jan 2007)

The configuration file listed in the above command is a hacked up version of the file available from this site.

So I type "make program" and my make setup knows to program.
When I want to debug, I run "make debug"
which does this:

debug: main
    openocd -f cortex_ft2232_dbg.cfg

It calls a different openocd config, and instead of automatically programming, it just connects to the jtag and leaves the usual openocd debug interface alive.

After all this, we finally get to the title of this post:
I run the command "arm-none-eabi-gdb main.out" which invokes the gdb from the Codesourcery toolchain (if you're using a different GNU toolchain, call the appropriate gdb). In this line "main.out" is the name of an elf formatted executable, created by "make program" built with the "-g" gcc flag.
I get a gdb prompt.

Now it's time to tell gdb how to communicate with my target:

 target remote localhost:3333

and what happens is this:

(gdb) target remote localhost:3333
Remote debugging using localhost:3333
bufused (buffer=0x2000081c) at circbuf.c:52
52	}
(gdb) 

and now we see that I've landed happily inside my code, buried somewhere in a circular buffer routine.
From my code, I know that I want to stop and examine variables in main.c at line 22, so I:

 (gdb) break main.c:22
Breakpoint 1 at 0x116: file main.c, line 22.
(gdb) 

and then I resume execution with "continue" wait for my breakpoint to kick in, and then start examining variables:

 (gdb) continue
Continuing.

Breakpoint 1, main () at main.c:22
22	        switch(c)
(gdb) disp c
1: c = 111 'o'
(gdb) 

If you've got a tool like ddd or some gui-fied gdb thing, you can probably proceed in a similar fashion.

Questions?