Archives For August 2009

>The timeline work I mentioned earlier has been going well. The patches are pretty well on their way. Until this lands, I am keeping the instrumentation patch unbitrotted each week.

On the “startup” team, we use the bugzilla whiteboard to mark all startup-related bugs with “[ts]”. I wanted to propose that in the meantime before the startup patch lands, feel free to mark [ft] in the whiteboard and list any function names you want to be timed with cold/warm start, platform, etc.

I will run the [ft] query often and add the timers to the instrumentation patch(es), and update the bug with the details.

Let me know if you have any ideas or questions.

>Using JSHydra to find unused functions in the Mozilla tree

JSHydra is a static analysis tool for Javascript written by Joshua Cranmer. I heard about it at one of Mozilla’s Monday foundation meetings, and it sounded like it might help me introspect JS similar to the way Python’s inspect module – which is a dynamic analysis tool.

Dietrich filed a bug ( 506128 ) about using JSHydra to find dead code in Firefox, especially in the startup code paths. I needed to shift gears at work, so he suggested I start using JSHydra.

There are a few tasks in order to find code via the static analysis route.

1. Build JSHydra

2. Build the preprocessed JS, XBL and XUL

(The XUL and XBL are stripped of markup and all javascripts left in the file as well as all event handlers which are defined as functions)

3. Generate the Javascript “context” by combining all Javascript files, XBL and XUL Javascript snippets for any Firefox chrome url or other type of context, Javascript modules for instance.

JSHydra operates on one file at a time, this is why all of this is needed.

4. Write Javascripts that use the JSHydra resultant AST and API to find all functions defined in a context, as well as all of the executed functions. The difference of these two lists will give us an idea of what functions may be “dead code”.

Let’s do this thing:

The first step is the get JSHydra built, which is built on top of Mozilla’s Javascript engine. This is a relatively easy task:

1. checkout my clone of the original JSHydra repo (http://hg.mozilla.org/users/Pidgeot18_gmail.com/jshydra):

hg clone http://bitbucket.org/daviddahl/jshydra-ddahl/

2. configure the build with 2 paths: –moz-src=path/to/mozilla/checkout and –moz-obj=/path/to/mozilla/object/dir

* The above configure options will use your existing Javascript build, otherwise, the makefile will clone mozilla’s JS source and take a bit longer to build.

*NOTE: I think there might be something wrong with the configure script, I had to manually add these lines to my config.mk:


MOZ_OBJDIR := /home/ddahl/code/moz/mozilla-central/obj-i686-pc-linux-gnu-optimize
MOZ_SRCDIR := /home/ddahl/code/moz/mozilla-central/mozilla

3. Run make.

This will build JShydra. To use JSHydra against a Mozilla-based project such as Firefox, you need to have all of the preprocessed Javascript built.

4. Edit jshydra-ddahl/myrules.mk:

Make sure JSTOOLDIR in myrules.mk points to /path/to/src/mozilla/config

copy jshydra-ddahl/build/ChromeHacker.py,
jshydra-ddahl/build/NicePreprocessor.py, and
jshydra-ddahl/build/myrules.mk
into /path/to/object/dir/mozilla/config

and also copy them into /path/to/source/mozilla/config

AND, copy myrules.mk into path/to/src/mozilla/js/src/config

(the js system uses a separate build system)

5. Do a full build of Firefox, and ‘make jsexport’

You need to run “make jsexport” on your object dir once Firefox is built.

This step should produce a directory in /path/to/obj/dir/dist/jsfiles

The jsfiles directory will have all of the “preprocessed js, XUL and xml (XBL) files.

6. context.py and jscontext.py

I have put together some python scripts that will attempt to build a Javascript file with all of the Javascript that is in a single JS context, i.e: chrome://browser/browser.xul – or – nsBrowserGlue.js

context.py will generate a context for a chrome URL.

jscontext.py will generate a context for a JS component or module.

The reason for this is that JSHydra only operates on a single file, so context and jscontext will try to package all of the JS into one file.


python context.py --help
python jscontext.py --help for operating instructions.

7. Finding dead code:

I have written a script using the JSHydra api to try and find functions that are not called but defined in a context, this will give us some idea of where to begin looking for code that is never executed, code that might need to be removed from the tree. It is a bit inaccurate, as I have not accounted for globally available functions like setTimeout, escape, unescape, etc. and functions that are part of xpconnect components. There are also features of JSHydra that are not complete that will make this more accurate.

The dead code work in progress is in scripts/deadCode.js, run it like this:

jshydra scripts/deadCode.js /path/to/newly/built/jsContext/file.js

And you get a lot of output as it builds up lists of functions both defined and called. At the end is a summary:


####################################################################
################ JSHydra *Possible* Dead Code Summary Report #######
################ JSContext: post-build/sample_browser_xul_context.js
1073 Defined Functions
503 Functions Called
105 Functions Called But NOT Defined in this context*
423 Functions named 'undefined' - (anonymous)

It has been pretty interesting doing JS static analysis, I hope more hackers will find this worthy of working on. You can find some helpful folks on irc in #static and #startup.

Special thanks to Joshua Cranmer, David Humphrey, Taras Glek and Dietrich Ayala

>I have been meaning to put up a post about our Firefox startup time work. Vlad and Dietrich have been leading some efforts in trying to identify and understand any bottlenecks that occur when Firefox starts up. This is not an easy thing to do, it seems we are need of more profiling tools.

Vlad has been hacking on a set of C++ and JS function timers for little while, and I also pitched in. The patch Vlad started was both a set of timers and a lot of instrumentation. He had me split it up, clean up the timers and change how it logs the timeline data.

The Timeline data looks like this:


> XRE_main
- 190 ms ( 190 ms total) - XRE_main [ScopedXPCOMStartup]
> NS_CreateServicesFromCategory: xpcom-autoregistration (start)
NS_CreateServicesFromCategory: prefservice:after-app-defaults (prefservice:after-app-defaults)
nsIOService::Init
> nsSocketTransportService::Init
- 0 ms ( 0 ms total) - nsSocketTransportService::Init [Created thread]
- 0 ms ( 0 ms total) - nsSocketTransportService::Init [UpdatePrefs]
NS_CreateServicesFromCategory: xpcom-autoregistration (end)
NS_CreateServicesFromCategory: xpcom-startup (xpcom-startup)
nsChromeRegistry::Init
- 0 ms ( 0 ms total) - nsChromeRegistry::Init [NS_RegisterStaticAtoms]
- 0 ms ( 0 ms total) - nsChromeRegistry::Init [GetProtocolHandler 'jar']
- 0 ms ( 0 ms total) - nsChromeRegistry::Init [A]
- 0 ms ( 0 ms total) - nsChromeRegistry::Init [B]
> nsChromeRegistry::CheckForNewChrome
- 4 ms ( 4 ms total) - nsChromeRegistry::CheckForNewChrome [processed chrome manifest /home/ddahl/code/moz/mozilla-central/obj-i686-pc-linux-gnu-optimize/dist/bin/chrome/browser.jar]
- 4 ms ( 8 ms total) - nsChromeRegistry::CheckForNewChrome [processed chrome manifest /home/ddahl/code/moz/mozilla-central/obj-i686-pc-linux-gnu-optimize/dist/bin/chrome/browser.jar]
- 0 ms ( 8 ms total) - nsChromeRegistry::CheckForNewChrome [processed all chrome manifests]
- 0 ms ( 8 ms total) - nsChromeRegistry::CheckForNewChrome [processed all skin manifests]
SetWindowCreator
- 0 ms ( 0 ms total) - SetWindowCreator [RegisterFactory done]
- 0 ms ( 0 ms total) - SetWindowCreator [Got ToolkitChromeRegistry service]
- 17 ms ( 17 ms total) - SetWindowCreator [OS Accessibility check]
...

There are still a few more iterations to go before this patch is checked into Mozilla Central, but I thought I would point it out to developers since it is quite generic and can time any functions in the Mozilla source, not just startup. While vlad, brenden, wan-teh and bsmedberg are iterating on review and “the hard stuff”, I will keep working on placing timers, hopefully with help from other knowledgeable Mozilla hackers.

Here is the latest wiki page on usage.

I am also working some visualization scripts and tools to make it quick to identify the bottlenecks. Right now, I am using MIT’s Simile Timeline. Pretty cool stuff.

I am hoping that some seasoned Moz Hackers might know of more places to insert timers, or perhaps contribute additional targeted timing instrumentation. The cool thing, of course, is that we added an ac_add_option to turn all of this on, otherwise it is noop code, making it pretty clean to use.

If you have any questions we are all hanging out in #startup. Also, send me an email: ddahl mozilla