Frontier Exclusive Visionary Interview for hardware, software, system related business and and academia
Frontier Journal (FJ): So Professor Patterson, I have a couple of questions for you. Let's start. First question is, what made you start RISC I project in the 1980s? Are there any insider stories that you want to share with us?
Prof. David Patterson (DP): Sure, I could do that. I had been working on a research project as a brand-new assistant professor. I'd just gotten my Ph.D. and I was working on a research project that probably didn't make a lot of sense. But I took a sabbatical and spent it at Digital Equipment Corporation, which was making a more complicated mini-computer.
And as a result of that experience-at Berkley we were trying to design single-chip computers, microprocessors-and I came back from that experience realizing it'd be very hard to build this back microprocessor, up until the microprocessor that was compatible with that destructive set, on a single chip. And so if it didn't make sense to build a back on a single chip, what did it make sense to build? And so I think that was-for me, that was my inspiration to work on the Risk project.
FJ: Okay. So at that time, the implementation technology was NMOS or CMOS?
DP: It was NMOS at the time.
But Intel was building microprocessors. By the time microprocessors were seen like toy computers in that you couldn't do real computing on them. So they were hobbyist computers and just the beginnings of the first-this is just the very first introduction of the PC was happening at that time.
FJ: Okay. Part of the RISC I project was instruction set design. How about that?
DP: Yeah, the emphasis at the time was to try and use quantitative principles to design an instruction set that would result in good performance. That was the impetus.
FJ: Okay. So from a physical technology perspective, Moore's Law still applies in the coming decades, most likely. Once semiconductor technology hits its dead end, what is the most promising technology for digital computing for physical implementation?
DP: I think it's risky to think that silicon technology will come to a dead end. I have a colleague, Chenming Hu, who was a professor of circuit design and now he's the Chief Technology Officer at TSMC, so he's a really world expert. He thinks things will slow down some, but he can imagine that some of the techniques that people are using to try and build carbon nanotubes or just nanotechnology in general, that can be applied to standard transistor technologies to make them shrink as well. So speaking as a computer designer, I think unquestionably the big opportunity is to figure out how to use lots of independent computers on a chip. So today, if I think about the risk designs you talked about-it was about 45,000 chips. We have enough resources to put 2,000 of those on a 65-nanometer chip, including caches and so on, but we don't know how to program it. So we've got the resources to do that. So I think what would make sense from a computer design perspective is to try and figure out how to create software and design hardware that can efficiently use 1,000 processors on a chip. If we can do that, then even if Moore's Law is to slow down, we could still figure out more performance. Or even if there is some other technology, it seems likely to me that it's going to need to take advantage of parallel computing. So my idea would be, even if Moore's Law is ending, there's an important problem to solve and whatever replaces C-MOS, I think we'll be able to take advantage of lots of simple computers rather than one big, complicated computer.
FJ: I understand. So what's the Rule of Thumb in designing a good instruction set? How does instruction set architecture interact with compiler technology as well as hardware architecture? Let's take RISC I as an example.
DP: In the past?
FJ: Yeah.
DP: A long time ago, people programmed primarily in assembly language. And there's still - But as programs got bigger, that just became infeasible to write whole programs in assembly language. As a result, the emphasis on instruction set design was efficiently matched to what compilers could produce rather than what humans could write, so that was a very important interface in the Risk Era. It didn't make sense to design instructions that your compiler can't use. Or it didn't make sense to design instructions that if the compiler were to generate them, weren't any faster than using simple instructions. So that was the idea behind it.
FJ: I see. So according to a presentation made by Professor Newton at Berkley, he listed eleven top R& D projects out of Berkley. Three of them were done by your group, namely RISC, RAID and NOW
DP: Oh, that's nice!
FJ: Why did you start to work on RAID technology after RISC I?
DP: Well, we thought at the time that the processor was going to get a lot faster, but it wasn't clear that I/O was going to improve as fast as the processing would. So that the speed of putting something into storage, if that didn't get any faster, then we'd spend all our time waiting for the storage system-even if processing got a lot faster. So after a while it wouldn't make any sense to have faster computers if the storage wasn't any faster. So that was my original idea. My colleague Randy Kantz had just bought a Macintosh desktop computer, and it came with a separate disk. He asked himself a question: "this small disk that sits next to my Macintosh computer is a very interesting piece of technology. I wonder what we could do with it?" So the combination of the opportunity of small disks and the need to create IO systems that could keep up with these faster risk processors led to the Rate Project. So it was originally seen as how to get more performance by replacing a big disk with lots of small disks.
FJ: Why are parallelism, scalability, and programmability the top three critical effects in high-performance computing? And also why do storage and communication also become important?
DP: Well, I think what's called supercomputing was originally focused just on the computation. Because I think that was the harder problem in the beginning, people focused just on that. Now as people wanted to solve real programs. not just do research papers on how to show how programs can be parallel and faster, big problems have big data as well as lots of computations. So once again, like for the original radar argument, if you just did the computation part much faster and didn't do anything the storage piece of it, you'd be limited by how fast you could do the storage. So that's what's happening in the high-performance computing is that they're focusing more on storage today than they did ten and twenty years ago. They're trying to get high-speed storage.
FJ: Okay. Yeah. So in terms of this parallelism, why parallelism exploitation is so important in high-performance computing? For example, from process level way down to operator level. We can use parallel languages for specifications or parallel compilers for parallelism extraction or other kinds of things. Even we can use hardware for parallelism dynamic scheduling and all kinds of things. Why is parallelism exploitation so important? And why does parallelism exploitation also have its limitations?
DP: Well, why it's so important is we don't know how to build a single computer that runs a single problem fast enough to solve the biggest problems in the world. If we knew how to do that, that would be a tremendous accomplishment.
If you think about today, for that to happen we would need something-right now, processors are running at Giga- hertz clock rate not very efficiently. We would need something that would operate at Tera-hertz clock rate and be more efficient than they are today. And since we can't do that, that's why we're forced to go to parallelism. And so instead of running a single program at an extremely high clock-rate, we instead run lots of things at the same time at a slower clock-rate. We're forced to do that. Now, in terms of what the natural limits are to parallelism, so far in attempts to find highly parallel programs, usually there are portions of it that seem to be inherently sequential, that it's very hard to [unintelligible] and make some parts of a real program parallel where you're just- An example would be just making a decision. You're looking at the data and you're deciding to try and go one of the four ways and you don't know until you get the data which way to go. Well, that's very hard to do that decision in parallel. When you're talking about thousands-of-way parallelism it doesn't take very much of these sequential pieces to end up being the bottleneck of going faster. So that's probably the limit to parallelism. What people have done to kind of overcome that is to solve much bigger problems. So if the problem you want to solve is kind of-no matter how big a problem, the problem is important. No matter how big it is, how much data you have, it's an important one to solve. That's what people have been doing. Solving bigger and bigger and bigger problems.
FJ: So it really is application-dependent.
DP: Yes. It's application-dependent. It'll also have to do with what happens if you scale the problem size. So it depends if you're trying to solve a problem or if you're trying to show how efficient your computer runs. If you're trying to solve a problem, if you're a scientist, you have a particular idea of a problem you want to solve. Probably it has so much data and I'd like it to be able to solve it overnight. Now what's going to happen, on the other hand, if you're worrying about how efficient you want your computer to run: this computer designer might take the size of his problem so his computer runs very efficiently. He might take a problem that's so big it would take a month to solve, but his computer would operate very efficiently in that time. Because sometimes when you double the data size, it takes four times as long or sixteen times as long to run. So kind of the problem there in terms of measuring computers when they talk about its efficiency (like in Tera-flops or eventually, Peta-flops) is well if the problem size is something the people care about or not.
FJ: Okay.
DP: Does that make sense?
FJ: It also looks like it depends on the problem's complexity.
DP: Right. Ideally, there's a linear relationship in execution time and data size, but there might be a quadratic relationship in computation time and data size. And so now, if instead of taking days or even a week it takes months or a year, well then that's a stupid size. Nobody cares about that.
FJ: Yeah. It depends on which algorithm we are using. So what's your comment on econ reason breaking news is IBM's newest processor achieving a 500-gigahertz operating frequency.
DP: I didn't read that article so I don't know what that is.
FJ: Oh, okay. I see.
DP: 500 Giga-hertz?
FJ: Yes.
DP: For a logic?
FJ: Yes. For a logic. It's a CPU-type something. One week ago.
DP: Did it talk about what technology it was?
FJ: It's the most advanced implementation technology. It's frozen. It's a very, very low temperature.
DP: Historically, very, very low temperature has been difficult because the question is, how do you repair these things?
DP: As long as it's running at low temperatures-something breaks: now what do you do? When you break it, now do you have to throw everything away? Can you save it? So to repair it-it's hard to repair something and keep it operating at very low temperatures.
FJ: That's one project we are working on. It's called ROC: the Rule of Computing.
DP: Yeah. That's something to do with super-cool logic, but yeah.
FJ: So you started the IRAM project after completing Louisville project in order to tackle the challenges of beta mismatch between microprocessor and memory. What is the Intel of your IRAM project on helping reconfigurable computing? To make it realistic?
DP: Reconfigurable computing? That wasn't so much the focus of what we were doing. The challenge for reconfigurable computing is first finding (again, like supercomputing), the applications that run well in reconfigurable computing. And right now, in reconfigurable computing FPGAs are the example of. It's very inefficient, both in area and in power. So you have to find an application that can overcome that inefficiency in area and power, and it's so much efficient by making it reconfigurable that can overcome that disadvantage in area to at least a factor of ten. So either somebody has to figure out a new way to make reconfigurable so that the disadvantage is smaller, or you have to find an application whose advantages factors of at least ten or a hundred to overcome that. Some examples of pattern matching and things, people have found examples. People have been looking at this for a while...
FJ: There are lots of start-ups, actually.
DP: Yeah. Over the years, there have been start-ups. It's got tremendous potential and so far, somebody either hasn't found the right way to program it, or maybe the right way to reinvent FPGAs, or find the killer application that everybody says, "Aha! That's a great use and we'll build that!"
FJ: So your point is that reconfigurable computing is far away from commercialization?
DP: What I would say is that it's been an idea that a lot of people have investigated for a while, and so far it's had modest success. It doesn't mean that somebody won't-somebody might still find it. But so far they haven't found it, the big killer application. So far it's been small successes, even though it's got tremendous potential.
FJ: FPGAs the relationship with reconfigurable computing. That brings a follow-up question. It was quite popular, like AOSI Logical and other companies working on structure ASICS design. Do you have any comments on this structure ASIC?
DP: Structure ASIC? Yeah. I think people are looking at it right now. I just don't know-the challenges for designing are the cost of math and the cost of CAD tools. So if structure ASIC where there are fewer customization layers-I think structure ASICS are trying to overcome some of those cost disadvantages, and this potentially means that CAD could be similar, too, with some of the restrictions there. So I think people are looking at a lot of ideas which will test your ethical principles. You know, what are you going to tell people? What are you going to say to customers? What are you going to say to employees? So it's highly educational. It'll be a test of your moral system when tempted by all of the dollars. And it's an important test because nine out of ten times, no matter what you say and do, you're going to go out of business. Right? So it can be a really interesting experience. Something you're proud that you've done. But you want to make sure you come out at the end proud of what you've done, whether you made money or not. So that would be my advice.
FJ: Yeah. Great insight. Professor Patterson, thanks for your time.
DP: Okay.
FJ: I highly appreciate it.
DP: All right.
FJ: Okay, bye-bye.
DP: Bye.
FJ: Bye.
DP: Bye.
Back to Home Page