------------------- HPSDR Recorder 0.74 ------------------- ------------------------- March 2012, Michael, N3UC ------------------------- The HPSDR recorder program is design to write samples from the Mercury board without any corruption from the windows audio system or VAC programs. Several file formats are supported as well as a scheduling/scripting system. With HPSDR Recorder you can setup a schedule to record to disk up to 4 receivers, the MIC data, and Wideband data from Mercury. As of version 0.5 the supported Mercury version is 3.1. Other version have not been tested and may or may not work. The program can use an Ozy or a Metis board. If using a Metis board, make sure that the 255.255.255.255 broadcast address is routed to the correct network card to find Metis. The code as of 0.74 will not broadcast to every network card in the system. Normally, if PowerSDR can see Metis so can HPSDR Recorder. One of the main uses of the program is to record data for FMT contests. The frequency displayed on the screen and used in the filenames is the indicated frequency. The error in the NCO tuning on Mercury is calculated and written to the SRI file. There is not an LO offset used between the tuned frequency and the displayed frequency. New Features ------------ 0.74 Added WAV file support, 16 bit, 2 channel Added PowerSDR file support, this is a WAV IEEE Float, 32 bit, 2 channel format Added Padding Event to turn on/off Metis padding Added 16-bit Shift to select which 16-bit are used from the 24 input bits for I and Q Added Shift16 Event to contol 16-bit Shift control Quick Startup ------------- The First item to do is a StartRadio. This will bring up a form to select the number of Receivers and SampleRate to start Mercury. The AutoInterface button will first check for an Ozy board and if it can't find on it will look for a Metis board. If you have problems finding a Metis board, check your routing table (run "route print") and look to see that the 255.255.255.255 destination is going to the correct network card. Make sure the correct card has the lowest Metric. You can add a static route if needed to correct the broadcast. With the Radio running you can use the PreSet buttons to tune frequencies or enter a frequency and use the Set Freq button. There is not currently a way to use the mouse wheel to tune. The program does not support audio output. It is designed to record I and Q samples and not to generate audio. With the frequency set and the Radio running you can now start writing files. The first step is to select a file format from the pull down. The Complex formats will write both I and Q while the Real formats will only output the I samples. The next step is to select the Write File button. This will open a file selection dialog box. Enter a BaseFileName (no extension). The program will create several files at this point. It will create a BaseFileName_SRI.txt file that contains information about the recording and BaseFileName_rx0.dat (more if you have more receivers) that contains the raw samples. The .dat files don't contain a header, the are just the samples written out. The Complex 24 format will contain all samples unchanged right from Mercury with the exception that as of Mercury 3.1, the spectrum from the FPGA is inverted (the Q samples have the wrong sign) so the Q samples are multiplied by -1 to fix the spectrum. The other formats will under go a simple casting to the data type used to write out. Radio Controls -------------- The -20dB checkbox will turn on or off the Atten on Mercury. Rx Ctrl will select which receiver the preset buttons will tune. Plotting Controls ----------------- Plot IQ will display the I and Q signals on the Receiver Displays. Yellow is I, Red is Q IQ Scale will scale the plot BaseLine is the value of the lowest values on the Spectrum display The pulldown below Baseline is the FFT windowing function. The Seq Skew, Block Skew will offset Wideband samples used to correct for the Wideband bug currently in Ozy and Metis. Zoom will zoom in the Wideband spectrum Plot Skew will skew the Wideband spectrum to the left Time Domain will show the A/D samples on the plot Show Block will display the Wideband block edges (used for debugging the wideband bug) File Controls ------------- 16-bit Shift will change which 16-bits of the input 24-bit I and Q values are written to disk. Normally 8 would be used to take the 16 MSB from the input data but better dynamic range can be achieved by selecting a lower value if not all 24-bits contain data. The output 16-bits will be calculated by: OutputI = InputI >> Shift16; OutputQ = InputQ >> Shift16; Write MIC Data will create a file and write the microphone data during recordings. Write Wideband data will create a file and write the wideband data during recordings. Scheduling ---------- Run Schedule will turn on or off the Scheduler ShowSched will open the Schedule editor form. The Epoch is always in GMT. A schedule contains an Epoch which is the starting time of the schedule and a list of events that happen a given amount of time after the Epoch. If the Run Schedule box is checked, a schedule will be running even if the Editor is open. The Now button can be used to change the Epoch to the current time. The schedule will start running right away. When the program starts it will look for a file called Events.xml in the application directory, if the file exists it will be loaded. There is a special case for the Epoch as saved in the xml files. If the Value of the Epoch is less than 0, then the Epoch will be set to (-1*Value + Current Time) when the file is loaded. This lets schedules be used as scripts and run right away when loaded. The Events.xml if set to a -1 Epoch will act like a startup script to control the start of the application. The Example button will just put some example events into the schedule. The schedule file is an XML file when can be edited by the user. The main reason to edit the file is to change the Epoch to -1 or to rearrange the order of the events. The Remove Used Events checkbox will remove events as they are run. This is useful for the AtomicFileChange and Status commands so the schedule does not become fill of the events they generate. This command will not remove events that expired before the checkbox was checked. They are only removed as they are run if the box is checked. Events ------ The main section of the schedule are the events that it can run. They can be used as scheduled events or scripting events (set the delta time to 0 and an epoch of -1). The program as of 0.73 has 18 commands. Commands contain a Delta time to run after the epoch (not after the previous command). The schedule will sort the commands in delta order. Most commands will take parameters in the form of a string. Currently there is not a lot of error checking for parsing of the parameters. The entry form contains example parameters for each event type. Command: Tune Params: Rx, Frequency This is used to turn a Receiver to a given frequency. Rx 0 is the first receiver. Frequency is in Hertz. -- Command: SetAtten Params: Value This is used to set the Attenuator to on or off. Value should be 0 or 1 -- Command: StartRecord Params: BaseFileName, fileformat Start recording. The BaseFileName can have the substitution stings like {DATETIME} etc. The fileformat is the Index into the format Pulldown box staring with 0. -- Command: StopRecord Params: NONE Stops the current recording -- Command: Message Params: Message String Displays a string in the Event status box -- Command: StartGnuRadio Params: Address, port Starting sending IQ samples to a UDP port in Gnu Radio format. The Address can be an IP address or hostname. The Port is the UDP port to send the data to. -- Command: StopGnuRadio Params: NONE Stop sending data to a UDP port -- Command: StartRadio Params: NumRx, SampleRate Start the Radio with the given number of receivers and the given sample rate. 0=48KHz, 1=96KHz, 2=192KHz. The interface will always be Auto Interface -- Command: StopRadio Params: NONE Stop the Radio Interface -- Command: LoadSchedule Params: Filename Loads a schedule and start running it. This is useful to chain schedules together. -- Command: SetPreSet Params: Num, Freq, Label Sets the label and Frequency for a PreSet button. Num is 0 to 3 for which button to set. Freq is the frequency in Hertz and label will be the text displayed on the button. -- Command: PreSet Params: Num Press a given preset button -- Command: SetMICWrite Params: Value Set the value of the Write MIC data checkbox. Value is True or False. -- Command: SetWBWrite Params: Value Set the value of the Write Wideband data checkbox. Value is True or False. -- Command: rxDisplay Params: NumRx Change the number of receivers to display. Set to 1,2,3 or 4 to change the display. -- Command: Status Params: StatusNumber, EventDelta This command will display the given receiver status and create a new event EventDelta from now if EventDelta is > 0. This is used to cause the status to be written to the SRI file every EventDelta seconds. -- Command: AtomicFileChange Params: BaseFileName, EventDelta This command will wait for a break in the incoming data then lock the file writing code, close the current files, open new files based on BaseFileName, then unlock the writing code and add a new AFC event EventDelta from now if EventDelta is > 0. This command is used to cycle files every EventDelta amount of time. The BaseFileName should contain a substitution string so the new files will not overwrite the old files. No data is lost between the file close and file open. -- Command: Padding Params: Value Turns on/off Metis missing sequence padding -- Command: Shift16 Params: Value Changes the value of Shift16 -- Command: CloseProgram Params: NONE This command is used to Close the program. This lets the program be started using Windows scheduler at a given time and it can be shut down when finished. File Name Parsing ----------------- In the filenames there are keywords that can be substituted when the files are created. All files will use: {TIME} {DATE} {DATETIME} {FORMAT} Only Data Files will use: {FREQUENCY} {SAMPLERATE} Example: BaseFilename is: WWV_{DATETIME}_{FORMAT}_{FREQUENCY} Files Created: WWV_20120305_161427_Complex24_{FREQUENCY}_SRI.txt WWV_20120305_161427_Complex24_10700000_rx0.dat Note that the SRI file will not substitute the {FREQUENCY} or {SAMPLERATE} values as these are attached to each receiver. (Currently all receivers will have the same samplerate but that might change later). The Frequency comes from the last data packet received and not from a commanded frequency, so if using substitution it is best to tune the radio and wait a second or two before recording so the filename will be correct. Spectrum Lab ------------ The data files can be read into SpectrumLab directly. But only Complex 16 and Real 16 seem to work well with SpectrumLab. The Spectrum Lab Server checkbox can be used to generate an Audio.dat file for realtime data to SpectrumLab but as of the current beta versions of Spectrum Lab they don't seem to read it correctly. Turning on the Server checkbox will generate a lot of data without much warning so it is not recommended at this time. Server files are tmp.dat and audio.dat in the current directory. Gnu Radio / UDP --------------- The Gnu Radio checkbox will open a form to ask for an IP address and Port number to send Complex Float I and Q values to a UDP port for inputing into Gnu Radio Realtime. Only one receiver should be running if sending to Gnu Radio. All receivers data will be sent in interleaved UPD packets. PowerSDR Support ---------------- To use a PowerSDR file from HPSDR Recorder in PowerSDR do the following: 1) Set PowerSDR to the same sample rate as used in HPSDR Recorder for the file. It will work if you have a different sample rate but the display might look funny. 2) Set PowerSDR to the frequency set on HPSDR Recorder for the file. 3) In PowerSDR, use the Wave File Controls to load the file and play it back. Make sure 1 and 2 are done before loading the file or the frequency will be incorrect. 4) After the file is running, you may tune PowerSDR around the spectrum etc. Issues / Notes -------------- As of 0.74 and Mercury 3.1: Wideband data from Ozy is in big Endian format while the Wideband data from Metis is in little Endian format. The Ozy data is converted to little Endian format before writing to the file. Wideband data from Ozy and Metis is not a continous stream of samples. There is an alignment issue that breaks up the samples. This can be seen in the KissKonsole as well. The problem can be corrected when using a Metis board with the controls next to the Display. When aligned correctly, the spectrum will not show leakage as it normally does (the signals will be nice sharp line). Mercury 3.1 (and all or most of the others versions) output the spectrum inverted. This shows up as the Q signal with an incorrect sign. The low level code in HPSDR Records corrects this by a Q = Q * -1, so the written files have an upright spectrum. This is the only change to the samples before they are written to disk. If comparing the I/Q plots to KissKonsole, the Q signal will have a different phase in HPSDR Recorder. The I/Q plotter uses the conventional Q lags I for positive frequencies. The PowerSDR file format requires an inverted spectrum so the Q is written as received from Mercury for this format. With Ozy there is no way to know if data is dropped in the current USB protocol. With Metis, the sequency number is checked and data can be padded with zeros if data is dropped (it is limited to 100 missing packets per drop in the code). This lets the recorded data stay coherent even with dropped data when using a Metis board.