Results tagged “testing” from dukeleto.pl

Parrot Embed Grant Update #5 : Zen Pebbles

|

I am still on the path of increasing test coverage in src/extend_vtable.c. It is much like a zen study, where you methodically concentrate on many tiny little pebbles, one at a time, moving them in the sand, to just the right place. According to the latest code coverage statistics, we are now at 72% code coverage, which is an increase of about 8% since my last report.

Many, many more tests involving Key PMCs were added. For an intro to what they are, take a look at my previous grant update. Many of the tests are clusters of related tests, because most VTABLEs have many similar forms which take integer, string or PMC-flavored keys. I ran into some platform-specific bugs which only manifest on Darwin machines, which were reported by Jim Keenan in TT# 2098 and which I then fixed by querying with a non-empty Key, which is more prudent.

I also ran into some actual bugs which I reported as Trac Tickets. First is that the cmp_pmc VTABLE does not seem to be working correctly from extend_vtable, which was reported in TT #2103. Then I fell into a "hole" in the VTABLE API, where ResizablePMCArray does not respond to defined_keyed(), which it should. This is described in TT #2094.

In retrospect, this was one of the most productive periods of my grant work. I estimate that I will be very close to the 95% milestone by my next grant update at this pace, which is very exciting.

Parrot Embed Grant Update #2

|

I have slowly but surely been increasing test coverage of src/extend_vtable.c, which when I started my grant was at 5% and is now sitting at 43%. I estimated that it would take about two weeks to get it to 50% and then another two weeks to get to 95%, but this was a bit optimistic. When I look back, giving myself a month for each would have been appropriate.

I do believe that my development velocity has increased recently, because I wrote a convenience funciton called extend_vtable_output_is(), which greatly reduces the number of lines of code that each extend_vtable test requires. The function is basically a wrapper around already existing functions that automates the process of compiling C source code use the Parrot C API, but it removes the need for roughly 40 lines of boilerplate stuff, such as including the proper header files, defining convenience functions that do basic error checking and data structure creation.

The whole reason this subsystem is undertested is because no function like extend_vtable_output_is() existed. Now each of my extend_vtable tests is around 10 lines, instead of 50. I've only had the new function for about half the time of the grant, so tests are now getting written much faster.

I have added about 700 lines of lines of tests since this grant started (using the spiffy new extend_vtable_output_is() function), and that has caused a 38% increase in test coverage. With the power of mathematics, we can figure out that (.38)*N = 700, which means N = 700/.38 = ~1842, where N is the number of lines of tests needed to cover 100% of the file (roughly).

My new estimate for getting to 95% coverage of extend_vtable is one month from now, Feb 24th, and then another two months for the rest of the grant.

The reason I say two months is that I am actually trying to hit a moving target, and the code coverage of extend.c and embed.c have actually *gone down* since I started the grant. Currently extend.c is about 6% lower and embed.c is about 14% lower. This means my grant is actually getting harder to complete.

I also learned about a very useful yet undocument environment variable called POSTMORTEM which makes Parrot test functions leave various intermediate files around for debugging purposes if it is set to true, which greatly helps in developing these tests. I plan to add documentation about this to the Parrot developer documentation.

Given the new test function and the new coverage numbers, I estimate that I will be able to complete this grant by late April 2011.

Clicky Web Analytics 42