BOINC Q&A — 09/30/06

Can you explain more on Average CPU efficiency and Result Duration Correction Factor? There seems to be some confusion about this, and little definite knowledge. For instance, some say a lower RDCF is better, others say an RDCF closer to 1.0 is best. Which is the truth?

CPU efficiency is the difference between how much CPU time a process received relative to the amount of wall clock time that has passed. It is the answer to the question of “In the last ten minutes or so, how much CPU did BOINC based science applications receive?” The thing to remember here is that the OS is constantly doing things in the background and each of those things eats a little bit of the CPU.

Duration Correction Factor is a per project value that measures the difference between the the expected time to process a result based on the benchmark verses what it actually took. A score of 1.0 means that the benchmark and the application processing time are in sync. The lower the score the greater the variance between what the benchmarks predict verse what it actually took to complete the result.

BOINC tries very hard not to ask for more work than it can actually process in a given period of time, so it tries to keep track of the machine overhead by the CPU efficiency score and Duration Correction Factor. Another thing to keep in mind is that memory speed plays a big part in the Duration Correction Factor. When you see similar processing times for a result for a 3.0Ghz processor and a 2.0Ghz processor it normally means that the 3.0Ghz processor is running with memory that cannot keep up with the processor. Or that both processors are bottlenecked with the memory speed.

We haven’t come up with a good solution for measuring the memory bandwidth problem yet. However, we are working on it.

BOINC version release notes do not seam as complete as they were before or am I looking in the incorrect places?

You can checkout the latest and greatest changes to BOINC at this web address:
http://setiathome.berkeley.edu/cgi-bin/cvsweb.cgi/boinc/

The file you’ll want to look at is ‘checkin_notes’ which contains the latest changes made to the client and sever packages.

You can see the check-in history for a specific branch by changing the tag specified near the bottom of the web page. The 5.6 branch tag is ‘boinc_core_release_5_6’. If you want to see the changes for 5.4 you would use ‘boinc_core_release_5_4’ and on it goes.

Any plans on releasing the full minutes of what went on (when your back), I read up on the 1st one but was a bit disappointed with the info on show it only gave a brief overview of what went on.

You can find the workshop proceedings here:
http://boinc.berkeley.edu/ws_06.php

How was the vacation?

I had a blast. I met a bunch of great people. I’m looking forward to going again next year.

wxWidgets 2.7 has been released. Is this going to be used in 5.6 or is it too late?

Too late for this release.

I’ve seen and myself tried to compile 5.4.x using Microsoft’s free Visual Studio Express 2005 editions (with all the bits and bobs needed, wxWidgets, SDK that needed..) Errors show up and does not compile. Is this fixed in 5.6, given this would probably be the major environment used by people trying develop BOINC under windows (since it’s free).

The BOINC DLL relies upon the ATL libraries which are not included in the express editions of the MS Development tools. I’m not sure if this is going to change in the future or not. I suspect that if we can incorporate a torrent library that doesn’t use COM/DCOM on Windows then I’ll invest more time into removing the need for ATL/COM/DCOM so that the DLL can be built with VS Express.

On a side note, I do not believe the express editions of the Visual Studio toolset contain the optimizing compilers or linkers. You might have to upgrade for those, or use the GCC toolset’s.

Would it be possible for you (since afaik you compile the final Windows releases) to put up instruction on how you compile BOINC. This may help a lot of people who just wish to dabble.

I’ll see what I can do.

How is the progress on low-latency-computing? Which projects expressed their interest in this feature?

I believe this feature was put in for a hospital who wished to be able to process MRI images faster than their current method. I’m not sure this feature will be used by a public project in the near future.

To submit questions for next week just click on the comments link below and submit your question.

Thanks in advance.

—– Rom

BOINC 2nd Pan-Galactic Workshop

This years workshop was very informative. It was really exciting to see the project leaders for some of the large projects as well as meet the project leaders for projects like PrimeGrid and Chess960@Home who started the projects as hobbies and who pay for them out of donations and out of their own pocket.

Here are a list of projects that I remember seeing right off the top of my head, sorry if I missed anybody:

  • BOINCStats Picture 008 Thumb1
  • GridRepublic
  • S@H
  • CPDN
  • PrimeGrid
  • Chess960@Home
  • LHC@Home
  • Africa@Home
  • malariacontrol.net
  • QMC@Home
  • SIMAP
  • R@H
  • WCG
  • Condor

On the first day, David gave an overall BOINC update of everything thast has been accomplished within the last year, since the first pan-galactic workshop and various projects described what they had been up too. People also had a chance to list out what topics of interest they had for break out sessions on the second day.

We wrapped up after the first day and went out to eat dinner at a fairly nice place to eat. I had to leave early since the motel I was staying at closed their gates at 11pm.

The next day we spent the morning in breakout sessions which included some of the following topics:

  • CreditsPicture 014 Thumb2
  • Grid Integration
  • Security
  • AMS Issues
  • Project Issues
  • Science Application Feature requests

I attended the sessions on security and project issues.

I can’t wait to see the summary role up for all the break-out sessions.

After lunch, we assembled and brain stormed about how to attract new members and new projects.

The last session of the day was for David and I to listen to everybodies wishlist of features. Some of which included the server-side code for CPU feature scheduling, torrent downloads, some more work refining our sample application, and others.

All-in-all it was a great event. Now we just need to come up with a plan to implement it all.

Catch you all later. I’m still on vacation and will return to my normal schedule after the 28th.

Picture 011 Thumb12

If you all want to see additional pictures from the conference you can go here.

—– Rom

BOINC Q&A — 09/22/06

I would just like to know more info about the kind of technical challenges you will face whilst attempting to port BOINC over to them?

At this time I really don’t know. Until I can get some documentation on the supported API sets it’ll be hard to give any answer. The only thing I’m pretty sure about is that the management interface will have to be written from scratch for each console.

How is the simple GUI coming on any ETA on this?

As far as I know it is almost feature complete. The guys at WCG are probably going to kickstart the beta process with their WCG beta testers since it looks as though we’ll still be putting the finishing touches on 5.6. I believe we’ll start to see clients with the BOINC Simple GUI (BSG) in a few weeks.

Does it work well from what you have seen?

Kevin Reed has been doing an awesome job polishing up the interface. I like what I see and this will give us a way to add more to the Advanced GUI without having to worry about confusing people.

How is the sand-boxing of BOINC on Windows coming along?

I haven’t started that work item yet. I probably won’t be able to start until we have completed our agreements with WCG.

When will 5.6 be released and what are the main improvements?

The best answer I can give about when it will be released is when it is ready. Improvements include a new CPU scheduler and work-fetch policy, CPU feature detection, and video card detection. This release will be the first time the BM will use GTK2 on Linux and sandboxing science applications on the Mac.

Will we ever get column sorting in BOINC Manager?

Someday.

Talk between BOINC projects and BOINC and vise versa seems minimal (and project to project). No-one seems to know what each other is doing, could there be an improvement of talk between them, like modifications to the servers, message boards, stats and developments in the pipeline.

I believe that is what the BOINC Workshops are about. We just finished the second one last week. It was productive and the projects gave us a bunch of work to do. I believe David is going to publish the results of the workshop within a week or two. I won’t know timeframe until after I return from vacation.

Where does BOINC need help?

Hereis a list of things we have identified that we would like to have. Contact David before starting anything though in case somebody else has already started the project. That list is likely to grow after the notes of the second workshop have been processed.

Can you explain more on Average CPU efficiency and Result Duration Correction Factor? There seems to be some confusion about this, and little definite knowledge. For instance, some say a lower RDCF is better, others say an RDCF closer to 1.0 is best. Which is the truth?

I’ll have to get back to you on that. I need to look over the code again.

BOINC version release notes do not seam as complete as they were before or am I looking in the incorrect places?

We are moving toward the firefox style of release notes. For a detailed list of changes you’ll need to lookup the checkin_notes for a specific release. I’ll follow-up in another post on how to do that.

Any plans on releasing the full minutes of what went on (when your back), I read up on the 1st one but was a bit disappointed with the info on show it only gave a brief overview of what went on.

I left the note taking to David and a few others, we’ll just have to wait and see what is published. I wouldn’t have been able to type fast enough to keep up with all the discussion and I still needed to be able to answer questions.

Sorry for being late with this weeks Q@A but I’m still on vacation. 🙂

To submit questions for next week just click on the comments link below and submit your question.

Thanks in advance.

—– Rom

BOINC Q&A — 09/15/2006

Well, since Martin was the only one to post any questions before I go to bed I guess the top ten will be his questions for the week.

When do we get to crunch on the new ALFA WUs?
When next for a reobservation run?
How well do the s@h-enhanced recrunched WUs compare to the s@h-classic results?

Ouch, these questions are best left to the S@H group. I had meant for this to be about the BOINC software itself since I really can’t speak for any of the projects.

I suppose I could attempt to clear up any confusion in what my roll is.

First off let me start out by saying that David A. manages multiple projects, of which S@H and BOINC are but two of them. I was hired to be a BOINC developer.

At the time I was hired bug investigations and fixes for SETIBOINC was divided up amongst the existing team which also had to manage SETI Classic. SETI Classic kept people pretty busy most of the time as people were running from one fire to the next. Under the existing model SETIBOINC was going to take awhile to finish up, so after discussing things over with David we decided that the best course of action would be to have me to head up the SETIBOINC migration until the migration was complete. SETIBOINC completed its migration in December 2005 and so my regular participation in the day to day happenings with S@H came to an end.

My role these days mostly involve helping David design and implement new features that will help drive further adoption by both new projects and new participants. I also manage the build, test, and release processes for BOINC.

These days I know just about as much as you all do about what S@H is up too. When David and I talk, in our daily phone calls, it is all about what is going on with BOINC.

What happened to utilising GPUs for processing WUs? (Are single precision floats good enough?)

Starting with BOINC 5.6 it will detect CPU features such as SSE and 3DNow as well as video cards. All the demand for video card detection has come from the community at large, so I don’t know of any project that is actively developing a client.

Whether single precision floats are good enough, that is up to the project.

I believe the biggest problem is actually the memory bus bandwidth between the systems main memory and the video card. Until PCI-Express was released the video card bus was design to push data to the video card as quickly as possible but didn’t do very well transferring data from the video card to system memory. It is my understanding that PCI-Express can handle the loads that DC apps will generate.

I suspect that once BOINC 5.6 is released we’ll be able to start publishing video card and CPU feature data to the stat sites and let them start generating some useful graphs for projects to look at. Once the projects know what is out there they might be able to justify the expense of specialized clients.

What of continued funding for Boinc and for s@h?

I can only speak about BOINC here, but I believe BOINC has secured funding for a couple of years.

Well, off to bed now.

—– Rom

BOINC Questions and Answers

Well, I’m back from my visit with family and friends. I leave again on Monday to attend the BOINC workshop in Geneva though.

After going over various pieces of feedback in this blog and in email it appears the general feeling is that we are not doing a very good job on the communication and outreach effort.

Starting this Friday I’ll try to answer the top ten questions posted to this blog entry. Every Friday after that I’ll try and answer the top ten questions posted to the previous Q&A post.

What do you all think?

—– Rom

BOINC, projects, account managers, and the community

What is up? It seems like there is some major disconnect between what David, I, GridRepublic, BAM, WCG, and the CPDN/BBC partnership have been up too and what the community believes is going on.

I don’t believe there are all that many people who actually believe that the current interface to BOINC is the best piece of software in the world and can be used by everybody on the planet. As a matter of fact, more often than not, we get feedback along the lines of “I wish it was as easy to use as S@H classic was.”

The biggest problem brought up at the First Pangalactic BOINC Workshop from the projects was the attach to project process.

For those who are relativity new out there BOINC used to require a project URL, and a authenticator in order to attach to a project. Authenticators are a string of 128 randomly generated characters. These were sent to you via email and had to be typed into the attach to project dialog which didn’t describe where that information came from or where to get it.

The authenticator turned out to be such a big problem that S@H didn’t want to complete the transition to BOINC until a better solution was in place.

There are two parts here:

  • S@H classic was a single executable with hard coded IP addresses, so the participants only needed to know their username to start processing data.
  • The user interface didn’t allow you to do anything, so you really couldn’t get into trouble.

In order to address these usability issues we first had to figure out what our goals ought to be. Here is a brief outline of our goals:

  • The stock BOINC software should not play favorites among the projects.
  • The stock BOINC software should not depend on any sever for normal operation.
  • The BOINC platform needs to be flexible enough for all research and business needs.
  • Find a way to simulate the simplicity of S@H classic’s attach process so the participant only needs to provide something easy to remember.
  • Find a way to simplify the interface.

One of the first proposals we talked about for Windows was a control panel applet that you could just check which projects you would want to participate in, which had several problems. Where was it going to get the list of projects? What happens if that server no longer exists? How do future projects get on the list?

Matt, of GridRepublic, approached us in late 2004 and wanted to see if he could help further BOINC’s adoption. He was especially interested in attracting the millions of users who might not otherwise join a distributed computing project because of ease of use problems. After a few sessions of brain storming Matt and David started to put together the concept of a web-based account management system.

At the time I was finishing up the last of the 4.x line of client software and helping E@H launch.

Support for the initial draft of the account management system began in early 2005 after the launch of E@H. By this time David and Matt started having regular phone conversations about feature sets and possible delivery times of features within the BOINC client software and the GridRepublic account manager. GridRepublic’s design goal became “provide an attractive, simple, and easy-to-use interface so that anybody could attach to any GridRepublic supported project with a click of a button.” an off-shoot of that was the need for a branded client so that the participant didn’t get confused about what they installed on their computer.

BOINC? What’s BOINC? I wanted to install this GridRepublic software to help fight cancer. Did I just get a piece of spyware? malware?

So the GridRepublic branded version of the BOINC client software was born to help reduce confusion for new participants. This had the added advantage that the GridRepublic client could be directly tied to the GridRepublic website without hurting the stock application. Usability studies, aren’t they grand?

WCG approached us early-2005 and asked if we would be willing to collaborate in allowing the BOINC client software run against the WCG servers. They were getting quite a bit of feedback from the distributed computing community that wanted to run WCG plus a few other BOINC projects. In turn they would help us come up with a simpler GUI and possibly other features that would be needed for corporate deployments. By mid-2005 IBM legal gave WCG the go ahead and work begun in earnest.

In November 2005, WCG launched the BOINC compatible interface to WCG. The amount of positive feedback they got was fantastic, the demand for a Windows based application was so high that they had to reschedule some other workitems to get the Windows version out the door.

In December 2005, we were approached by CPDN and the BBC to help put together a BBC branded BOINC client to run an experiment that the BBC was going to document. Working with the BBC was an experience, the visuals changed rather regularly but they did get better. We managed to complete the branded client by the deadline for the first part of the TV documentary in February 2006. The BBC did a few usability studies by pulling people off the street, a few things came from that including renaming “credit” to “work done” as some of the people polled were concerned that their credit card might be involved.

May 2006, Willy de Zutter of BOINCStats fame launched BAM.BAM largely flew under my radar as I don’t recall Willy asking for any client-side changes. I do recall a few bug reports being posted to boinc development though.

June 2006, WCG told us that they were ready to begin the simple GUI work, they had recently hired an internal developer who would do the work. Later that month we had mock-ups sent to us via email and were giving them feedback on what we thought. They started writing code at the beginning of this month.

Collaboration continues between BOINC, GridRepublic, BAM, WCG, and the community with tweaks to the UI to make things more easily understood by new participants. Account management systems continue to evolve as well, I believe the ultimate goal of the system is to provide a single remote interface to all of your BOINC based clients. Suspending/Resuming tasks, projects, and clients. Changing preferences for clients. Attaching and detaching to various projects for clients. In short anything you can do in the advanced GUI you could do via an account manager.

As far as the whole branding thing, I tend to look at the BOINC technology stack in a layered way. BOINC sits at the very bottom, projects sit on top of BOINC, and account managers sit on top of projects. Account manager’s need to be able to differentiate amongst themselves. Right now that consists of website design and client graphics. That doesn’t make them any less BOINC enabled than if they kept the stock graphics or website design.

Another analogy in the software world with branding is Linux. The Linux kernel sits at the bottom, applications sit in the middle, and distro’s at the top. Does Fedora get lambasted for changing the “start” button from the default KDE/Gnome button to a red hat? Would Red Hat get beat up if they said “use our software, x number of people can’t be wrong.” where x is the total number of Linux users worldwide when the target audience for the advertisement is Windows?

I think the hostility towards WCG and GridRepublic is unwarranted and harmful to BOINC overall. From where I’m standing, everybody is being a good citizen. I do not believe anybody is out to do BOINC or the community harm, so if you see a problem just drop them or me an email. Everything will get sorted out.

—– Rom

BOINC Workshop and Vacations

So I’m starting to get prepared for various trips in September.

For the first week in September I’ll be traveling to Anchorage, Alaska to meet my new niece. My sister gave birth to her first child a couple weeks ago, so this will be something of a new experience for me. On the 6th I’ll be leaving Alaska and head to Florida to check out my new diggs which I’ll be moving to at the end of November. On the 11th I’ll be returning home for a week. I bought some toys for my nieces and nephew, holy cow, toys are expensive.

September 18th I leave for Geneva, Switzerland to attend the The 2nd Pan-Galactic BOINC Workshop. I’m looking forward to meeting several of the scientists of the European BOINC based projects. I found out today that David Kim of R@H is going to go to the conference, so we are trying to sync up for the long flight over. My grandparents are going to meet me in Geneva and after the conference we are going to go tour around for a bit. Right now, I really don’t know where I’m going or where I’m going to stay, it is going to be something of a surprise. My grandparents are a lot of fun to travel with and I’m really looking forward to it.

On the 28th I head back state side and recoup. I’m sure the next couple of days after that will be spent relearning all the things I will have forgotten about BOINC. That is normally the sign of a really great vacation.

I have turned off various forms of article tracking for the blog that spammers like to try and abuse, so the favorites section won’t change while I’m away. Sorry about that, but I don’t know what my Internet connectivity situation will be like next month and I really don’t want to come back to a bunch of auto insurance, and poker advertisements to clean up.

—– Rom

BOINC and the gaming consoles

A very common request we have from the community is to be able to run BOINC and various projects on a gaming console, whether it be the PS3, XBOX360, or Wii.

As many have heard Folding@Home has announced that they have a client package for the PS3.

This is great news for all distributed computing projects, personally I think this will begin the domino effect of having all the console manufacturers want to be able to support this kind of development.

We had gotten some push back from one of the manufacturers about doing something like this at the begining of the year and since we had the stuff with the CPDN/BBC to do and then stuff with WCG we decided to put the console issue on hold. We are almost done the the WCG workitems and the CPDN/BBC stuff is already done.

When Microsoft announced XNA Game Studio Express I applied to participate in the Beta. Now I need to get ahold of the SDKs for the PS3 and Wii.

People should realize however, this is only half the battle. Once BOINC is ported, we’ll have to get a few projects running on a console.

This kind of adds another diminsion to the whole notion of console wars. I mean, can you imagine one of the administrative staff at a university getting a request to purchase 5,000 PS3s? I would really love to be in that office when that request hits their desk.

—– Rom