Lambda Delay TunerStudio Log Parser

[luftvagon]

Got bored, decided to write a "Lambda Delay" estimator using parsing of your TunerStudio logs. The more records, the longer the log, the better.

If anybody has a very long TunerStudio log, and you think your Lambda Delay is correct, please let me take a look at your log file to see if numbers will match close to the script output.

Usage:

Code: Select all

$ l_delay.gawk Downloads/2016-12-27_18.01.04.msl 
F:1656:S:1657:T:1662 Delay: 387ms	RPM: 1180	MAP: 64
F:1657:S:1658:T:1662 Delay: 322ms	RPM: 1196	MAP: 64
F:1666:S:1667:T:1670 Delay: 200ms	RPM: 2128	MAP: 65
F:1680:S:1681:T:1686 Delay: 334ms	RPM: 1502	MAP: 65
F:1719:S:1720:T:1724 Delay: 270ms	RPM: 2093	MAP: 65
...

Requirements:

• GNU Awk

Limitations:

• Single Wide Band Sensor

• Single Channel Batch Fuel Delivery

Change Log:

• 00001a

Code: Select all

Split "events" ahead into PW, and AFR. 
Spit out event line # for first,second,and third data point for log reference lookups.

• 00001

Code: Select all

Initial Release

[luftvagon]

Code:

Code: Select all

#!/usr/bin/awk -f 
# awk script to figure out lambda delay based on changes to pw, followed by afr changes logged within tunerstudio mlv logs.
# version 00001a

BEGIN {
	# Field Separator; either \t or , for CSV
	FS="\t"

	# How many PW events ahead to look for deltas
	pw_events_ahead=1

	# How many AFR events ahead to look for deltas
	afr_events_ahead=5

	# Reject events with CLT below
	clt_reject_below=170

	# Delta PW (10%)
	delta_pw = 90

	# Delta AFR (15%)
	delta_afr = 85
}

function cmp_num_idx(i1, v1, i2, v2)
{
     # numerical index comparison, ascending order
     return (i1 - i2)
}

/Time/ {
q=split($0,a,"\t")
for (v=1;v<=q;v++) {
	c[a[v]]=v
	}
}

/^[0-9]/ {
    q=split($0,a,"\t")
    track[NR]["TIME"] = a[c["Time"]]
     track[NR]["RPM"] = a[c["RPM"]]
     track[NR]["MAP"] = a[c["MAP"]]
     track[NR]["AFR"] = a[c["AFR"]]
      track[NR]["PW"] = a[c["PW"]]
     track[NR]["CLT"] = a[c["CLT"]]
}

END {
    PROCINFO["sorted_in"] = "cmp_num_idx"
    for (first_data_point in track) {
	if (track[first_data_point]["CLT"] >= clt_reject_below) {
		for (second_data_point = first_data_point; second_data_point <= first_data_point+pw_events_ahead; second_data_point++) {
			if ((track[second_data_point]["PW"] > delta_pw/100) && (track[first_data_point]["PW"] > 100/delta_pw) ) {
			    deltapw = track[first_data_point]["PW"]/track[second_data_point]["PW"]
			} else {
			    deltapw = 1
			}
			if ((deltapw < delta_pw/100) || (deltapw > 100/delta_pw)) {
			    for (third_data_point = second_data_point; third_data_point <= second_data_point+afr_events_ahead; third_data_point++) {
    				if ((track[second_data_point]["AFR"] > 0.1) && (track[third_data_point]["AFR"] > 0.1) ) {
					deltaafr = track[third_data_point]["AFR"]/track[second_data_point]["AFR"]
				    } else {
					deltaafr = 1
				    }
				    if ((deltaafr < delta_afr/100) || (deltaafr > 100/delta_afr)) {
					if ((track[third_data_point]["TIME"]-track[second_data_point]["TIME"] < 1 ) && (track[second_data_point]["RPM"] > 0)) {
					    printf "F:%d:S:%d:T:%d Delay: %dms\tRPM: %d\tMAP: %d\n", first_data_point, second_data_point, third_data_point, (track[third_data_point]["TIME"]-track[second_data_point]["TIME"])*1000 , track[second_data_point]["RPM"] , track[second_data_point]["MAP"]
				    }
				    break
				}
			}
			break
		    }
		}
	    }
     }
}

Alternative:

Find 0 PW, and seek for "freeair" AFR; display delta between the two events.

Code: Select all

#!/usr/bin/awk -f 
# awk script to figure out lambda delay based on changes to pw, followed by afr changes logged within tunerstudio mlv logs.

BEGIN {
	# Field Separator; either \t or , for CSV
	FS="\t"
	
	# Maximum events to seek ahead
	events_ahead = 15

	# Controller Freeair AFR
	freeair_afr = 22.3

	# Controller Power-on AFR
	poweron_afr = 7.5

	# Reject events with CLT below
	clt_reject_below=120
}

function cmp_num_idx(i1, v1, i2, v2)
{
     # numerical index comparison, ascending order
     return (i1 - i2)
}

/Time/ {
q=split($0,a,"\t")
for (v=1;v<=q;v++) {
	c[a[v]]=v
	}
}

/^[0-9]/ {
    q=split($0,a,"\t")
    track[NR]["TIME"] = a[c["Time"]]
     track[NR]["RPM"] = a[c["RPM"]]
     track[NR]["MAP"] = a[c["MAP"]]
     track[NR]["AFR"] = a[c["AFR"]]
      track[NR]["PW"] = a[c["PW"]]
     track[NR]["CLT"] = a[c["CLT"]]
}

END {
    PROCINFO["sorted_in"] = "cmp_num_idx"
    for (first_data_point in track) {
	if ((track[first_data_point]["CLT"] >= clt_reject_below) &&  (track[first_data_point]["AFR"] > poweron_afr)) {
	    if (track[first_data_point]["PW"] < 1) {
		found = 0
		for (second_data_point = first_data_point+1; second_data_point <= first_data_point+events_ahead; second_data_point++) {
		    if ((track[second_data_point]["AFR"] == freeair_afr) && (track[first_data_point]["AFR"] < freeair_afr) && (track[first_data_point]["AFR"] > poweron_afr))  {
			if (found == 0) {
			    printf "Event: %d-%d \tDelta: %dms \tPW: %3.2f \tMAP: %d \tRPM: %d \tAFR: %3.1f \t| \tPW: %3.2f \tMAP: %d \tRPM: %d \tAFR: %3.1f \n" , first_data_point, second_data_point, 1000*(track[second_data_point]["TIME"]-track[first_data_point]["TIME"]) , track[first_data_point]["PW"], track[first_data_point]["MAP"], track[first_data_point]["RPM"], track[first_data_point]["AFR"], track[second_data_point]["PW"], track[second_data_point]["MAP"], track[second_data_point]["RPM"], track[second_data_point]["AFR"]
			    found = 1
			}
		    }
		}
	    }
        }
    }
}

[MarioVelotta]

If anybody has a very long TunerStudio log, and you think your Lambda Delay is correct, please let me take a look at your log file to see if our numbers match. I'll be posting the script in the 2nd thread, after I do some more testing.

Hey Luftvagon, way to go! Can you teach me how to code?

How would our numbers match? Each install is going to be different due to countless differences from engine to engine. I've got some long logs with data from the 6 02 sensors I have in my exhaust right now The Major 4 are just off the heads about 4", the 5th is just before the collector on a type 1 sidewinder type exhaust and the 6th is about 4" further but after the collector. This was actually to test the internal lambda controller in my new ECU's against the LC2 that was in the 5th position.

[luftvagon]

Hey Mario,

The number(s) to compare would be your lambda delay (probably calculated using a different method/guessing/or default) versus what the your log produces. I don't have any of my own logs to try it on, so I will go for a long drive on Tuesday. Currently, I was testing this on someone else's 5 MB log file, and it got some good data points.

Here is what the program/parser does:
- I take each event in the log that has CLT>130F (adjustable) (event1)
- For each event, I look for subsequent events that have PW delta __ % (configurable) (event2)
- and then, I follow event2 subsequent AFR delta __%. (event3)
- If event3-event2 AFR delta % is met, spit out event3-event2 time delta, followed by event2's RPM, and event2 MAP.

In theory, this sounds "close" to the magic numbers we need. One could go further to average this, because there would be many data points.

[luftvagon]

I wrote another version that will only look for PW = 0, and then look for subsequent AFR DROP -- it may be more helpful:

Code: Select all

$ l_delay_3.gawk Downloads/2016*.msl | more
Event: 1812-1813 	Delta: 62ms 	PW: 0.00 	MAP: 17 	RPM: 2156 	AFR: 16.3 	| 	PW: 0.00 	MAP: 17 	RPM: 2149 	AFR: 22.3 
Event: 2153-2154 	Delta: 78ms 	PW: 0.00 	MAP: 17 	RPM: 2054 	AFR: 21.4 	| 	PW: 0.00 	MAP: 18 	RPM: 2023 	AFR: 22.3 
Event: 2542-2543 	Delta: 93ms 	PW: 0.00 	MAP: 18 	RPM: 1937 	AFR: 20.0 	| 	PW: 0.00 	MAP: 18 	RPM: 1950 	AFR: 22.3 
Event: 2629-2631 	Delta: 125ms 	PW: 0.00 	MAP: 22 	RPM: 1499 	AFR: 22.3 	| 	PW: 0.00 	MAP: 25 	RPM: 1451 	AFR: 8.4 
Event: 2630-2631 	Delta: 61ms 	PW: 0.00 	MAP: 23 	RPM: 1488 	AFR: 22.3 	| 	PW: 0.00 	MAP: 25 	RPM: 1451 	AFR: 8.4 
Event: 2631-2632 	Delta: 78ms 	PW: 0.00 	MAP: 25 	RPM: 1451 	AFR: 8.4 	| 	PW: 1.58 	MAP: 28 	RPM: 1470 	AFR: 7.5 
Event: 3601-3602 	Delta: 62ms 	PW: 0.00 	MAP: 17 	RPM: 2163 	AFR: 21.4 	| 	PW: 0.00 	MAP: 18 	RPM: 2154 	AFR: 22.3 
Event: 10211-10212 	Delta: 62ms 	PW: 0.00 	MAP: 15 	RPM: 2557 	AFR: 22.2 	| 	PW: 0.00 	MAP: 15 	RPM: 2565 	AFR: 22.3 
Event: 11536-11537 	Delta: 62ms 	PW: 0.00 	MAP: 18 	RPM: 1887 	AFR: 22.0 	| 	PW: 0.00 	MAP: 19 	RPM: 1880 	AFR: 22.3 

[Piledriver]

Nice work!

I'll give it a try later this week, waiting on some brake bits.

[MarioVelotta]

When it looks for 0.00 PW, are you then looking for the Lambda Feedback to reach freeair or just change from current?

[luftvagon]

Mario, I can make it either or configurable.. What would be freeair? 22.3? Guess that value would be controller specific.

awk code posted above.

[MarioVelotta]

Anyway I can run this on a PC without linux?

[Piledriver]

Anyway I can run this on a PC without linux?

More work than you would probably like.
Installing the gnu tools compiled for windows has always been iffy, if available at all.

That's why MS is working on a compatibility layer and has ported bash to WinX...
bash makes powershell seem like command.com...

Boot from a USB stick of DVD, recent Knoppix DVD has all of the tools you need, and you can "install" more packages w/o installing to you HDD. You can even install TS etc. Works fine, even with just a "persistent" Live install to USB.

You can even install fully to a large USB stick or drive, works fine, although Knoppix isn't really intended to be a real distro.
Mint 18/kde or Mageia 5.1 are probably better choices, and can also be installed to USB.
(Mageia 6 will be out RSN)

[luftvagon]

http://gnuwin32.sourceforge.net/packages/gawk.htm

[RHough]

Adding a self learn Lambda delay function is something Tuner Studio should be doing. All the information is there just need TS to look at the data stream and create a Lamda Delay map to work with VE auto tune. Parsing logs either manually or with a third party app is lots of work to make the auto tune function work correctly. I think what we need to look at is PWdot vs Lambda Response. Seeing auto tune adjust the same cells multiple times is odd. I assume that this behaviour is dependant on Lambda delay? Looking at PW = 0 and lambda response is a closed throttle delay time. When I look for the PW 0 to + and lambda response from full lean I get different numbers than when I look for PW + to 0 and lambda response. I think (dangerous) that when the delay is correct I should see longer periods of no EGO corrections? Having to set reference points in the fuel map to then parse logs looking for response times and guestimate the delay table is very time consuming. IIRC my WB samples at 12hz so output is every 83.3ms? I assume length from exhaust valve to O2 sensor is one factor, exhaust system volume to O2 sensor is another, and volume of exhaust at RPM & load is the last. What we need is to know what the exhaust volume is then calculate the number of engine cycles it takes for that volume to reach the sensor. That should give an RPM based first approximation of lambda delay? Then there is a +/- timing error from the lambda sample rate? Seems to me that =/- 83ms is as close as I can get?

[Piledriver]

At least the table in tunerstudio is only 3x3.

MS3 has a much larger delay (8x8?) table built into the firmware that needs populated.

The main thing is get it reasonably close, combustion itself has a lot of chaos, so there will always be noise.

The "default" delay values are for an LSx v8 with short cast iron manifolds with the O2 sensor a few inches from an exhaust port, so any workable boxer engine exhaust with collector mounted sensor will likely have a LOT more delay.

~Anything reasonably close will bring an improvement in VEAL performance.

[RHough]

At least the table in tunerstudio is only 3x3.

MS3 has a much larger delay (8x8?) table built into the firmware that needs populated.

The main thing is get it reasonably close, combustion itself has a lot of chaos, so there will always be noise.

The "default" delay values are for an LSx v8 with short cast iron manifolds with the O2 sensor a few inches from an exhaust port, so any workable boxer engine exhaust with collector mounted sensor will likely have a LOT more delay.

~Anything reasonably close will bring an improvement in VEAL performance.

LOL ... I built a spreadsheet.
Calculated volume from exhaust valve to O2 sensor.
Calculated time to fill exhaust volume @ 100% fill for 3 RPM points.
Made a *huge* assumption that VE table is representative of % cylinder fill (95 VE = 95% fill)
Built a table using 95, 70, & 45 load at 1000, 3000, & 6000 RPM and plugged it in ... seems reasonable...

I think I can get closer using the available data. Fuel volume should be a linear function of PW. Combined with AFR that should give intake charge volume. Loading a log into a spreadsheet to filter for RPM and load points should then allow calculation of intake charge volume to plug back into the delay spreadsheet ... "reasonably close" is an low bar to set.

[Piledriver]

The first time I did it, I took advantage of a bug in one of the early MS2E sequential firmwares that caused random single cycle fuel or ignition dropouts... it turns out my ancient JAW 1.03 WBO2 responded fast enough to see that even at 6K.
Took a bit of trolling around in the logs but it was frequent enough I got good delay data.

When I set my MS3 up, the delays changed a bit seemingly due to how the MS3 processes the data.
I made some temporary mods to the VE table at 3 RPMs with narrow RPM bins for extra fuel with minimal blend.
That worked too.

The MS3 in-memory table is 12x12 so that could take time to manually populate.
A tool either in the firmware itself or in TS is really needed.

I plugged the 3x3 table I use in VEAL into the 12x12 MS3 table and simply extrapolated everything else.
MS3 can use th 12x12 table for EGO correction, the closer it is to correct, the better it can work.