#Hello From now on, i will put my blog about coding here.
Mit 6828 lab1 ex1-7 booting
#Lab1
这个Lab干什么
- 了解Linux启动流程。
- 通过具体代码trace启动指令.
##Exercise #1 学习，了解ATT汇编语法。
- 略
##Exercise #2 学习， 了解C语言。 特别是指针
- 略
##Exercise #3
Take a look at the lab tools guide, especially the section on GDB commands. Even if you’re familiar with GDB, this includes some esoteric GDB commands that are useful for OS work.
- 略
Set a breakpoint at address 0x7c00, which is where the boot sector will be loaded. Continue execution until that breakpoint. Trace through the code in boot/boot.S, using the source code and the disassembly file obj/boot/boot.asm to keep track of where you are. Also use the x/i command in GDB to disassemble sequences of instructions in the boot loader, and compare the original boot loader source code with both the disassembly in obj/boot/boot.asm and GDB.
- 0x7c00是bootloader读取的第一条指令的地址。
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
obj/boot/boot.asm
obj/boot/boot.out: file format elf32-i386
Disassembly of section .text:
00007c00 <start>:
.set CR0_PE_ON, 0x1 # protected mode enable flag
.as
.globl start
start:
.code16 # Assemble for 16-bit mode
cli # Disable interrupts
7c00: fa cli
cld # String operations increment
7c01: fc cld
# Set up the important data segment registers (DS, ES, SS).
xorw %ax,%ax # Segment number zero
7c02: 31 c0 xor %eax,%eax
movw %ax,%ds # -> Data Segment
7c04: 8e d8 mov %eax,%ds
movw %ax,%es # -> Extra Segment
7c06: 8e c0 mov %eax,%es
movw %ax,%ss # -> Stack Segment
7c08: 8e d0 mov %eax,%ss
...
Trace into bootmain() in boot/main.c, and then into readsect(). Identify the exact assembly instructions that correspond to each of the statements in readsect(). Trace through the rest of readsect() and back out into bootmain(), and identify the begin and end of the for loop that reads the remaining sectors of the kernel from the disk. Find out what code will run when the loop is finished, set a breakpoint there, and continue to that breakpoint. Then step through the remainder of the boot loader.
- 继续追踪bootloader的启动流程， boot.结束之后会调用boot/main.c里bootmain函数.
- 阅读boot/boot.S可知, 最后执行的语句是
1
2
3
4
call bootmain
7c45: e8 c2 00 00 00
call 7d0c <bootmain>
//we can also verify that 0x7d0c is the address of function bootmain in boot/main.c
##Exercise #4
阅读boot/main.c的代码.
- Load Address(LMA) & Link Address(VMA)
- 首先用objdump查看kernel elf文件.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
objdump -h obj/kern/kernel
obj/kern/kernel: file format elf32-i386
Sections:
Idx Name Size VMA LMA File off Algn
0 .text 00001a47 f0100000 00100000 00001000 2**4
CONTENTS, ALLOC, LOAD, READONLY, CODE
1 .rodata 00000714 f0101a60 00101a60 00002a60 2**5
CONTENTS, ALLOC, LOAD, READONLY, DATA
2 .stab 0000390d f0102174 00102174 00003174 2**2
CONTENTS, ALLOC, LOAD, READONLY, DATA
3 .stabstr 000018a2 f0105a81 00105a81 00006a81 2**0
CONTENTS, ALLOC, LOAD, READONLY, DATA
4 .data 0000a300 f0108000 00108000 00009000 2**12
CONTENTS, ALLOC, LOAD, DATA
5 .bss 00000644 f0112300 00112300 00013300 2**5
ALLOC
6 .comment 00000027 00000000 00000000 00013300 2**0
CONTENTS, READONLY
- 一般来说, Load Address(LMA) 是程序(代码/.text)在内存中的位置, 计算机,期望从Link Address(VMA) 执行(代码/.text)
-
大部分时候, 两者是一样的. 关于这儿的0xf0100000和0x00100000, 在后边会有解释.
- 预备知识
- ELF文件格式 , 重点知道elf结构体中几个属性的意思。
- bootmain主要只做了这么几件事
- 读取kernel ELF
- 根据kernel ELF获取kernel的载入地址 (Load Address) , 并载入.
- 载入kernel之后， 根据ELF头获取kernel需要执行的函数地址， 并执行.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
#define SECTSIZE 512
#define ELFHDR ((struct Elf *) 0x10000) // scratch space
void readsect(void*, uint32_t);
void readseg(uint32_t, uint32_t, uint32_t);
void
bootmain(void)
{
struct Proghdr *ph, *eph;
// read 1st page off disk
readseg((uint32_t) ELFHDR, SECTSIZE*8, 0);
// is this a valid ELF?
if (ELFHDR->e_magic != ELF_MAGIC)
goto bad;
// load each program segment (ignores ph flags)
ph = (struct Proghdr *) ((uint8_t *) ELFHDR + ELFHDR->e_phoff);
eph = ph + ELFHDR->e_phnum;
for (; ph < eph; ph++)
// p_pa is the load address of this segment (as well
// as the physical address)
readseg(ph->p_pa, ph->p_memsz, ph->p_offset);
// call the entry point from the ELF header
// note: does not return!
((void (*)(void)) (ELFHDR->e_entry))();
// SUNUS: call this function and enter the kernel.
bad:
outw(0x8A00, 0x8A00);
outw(0x8A00, 0x8E00);
while (1)
/* do nothing */;
}
- 在
1
((void (*)(void)) (ELFHDR->e_entry))();
1
2
((void (*)(void)) (ELFHDR->e_entry))();
7d63: ff 15 18 00 01 00 call *0x10018
- 在0x7d63处下断点，之后si执行一条指令。正是 kern/entry.S 的第一条指令。 也标志着我们进入了kernel :), 通过查看obj/kern/kernel文件, 我们知道了进入kernel后的第一条指令位于 0x0010000c 处, 并且对应的文件为 kern/entry.S
1
2
3
4
5
obj/kern/kernel: file format elf32-i386
architecture: i386, flags 0x00000112:
EXEC_P, HAS_SYMS, D_PAGED
start address 0x0010000c
##Exercise #7
Use QEMU and GDB to trace into the JOS kernel and stop at the movl %eax, %cr0. Examine memory at 0x00100000 and at 0xf0100000. Now, single step over that instruction using the stepi GDB command. Again, examine memory at 0x00100000 and at 0xf0100000. Make sure you understand what just happened.
What is the first instruction after the new mapping is established that would fail to work properly if the mapping weren’t in place? Comment out the movl %eax, %cr0 in kern/entry.S, trace into it, and see if you were right.
-
这就很简单了,
1
movl %eax, %cr0
-
打开kernel的反汇编文件 obj/kern/kernel.asm, 得到打开内存应试的指令的地址为 0xf010025
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
obj/kern/kernel: file format elf32-i386
Disassembly of section .text:
f0100000 <_start+0xeffffff4>:
.globl _start
_start = RELOC(entry)
.globl entry
entry:
movw $0x1234,0x472 # warm boot
f0100000: 02 b0 ad 1b 00 00 add 0x1bad(%eax),%dh
f0100006: 00 00 add %al,(%eax)
f0100008: fe 4f 52 decb 0x52(%edi)
f010000b: e4 66 in $0x66,%al
f010000c <entry>:
f010000c: 66 c7 05 72 04 00 00 movw $0x1234,0x472
f0100013: 34 12
# sufficient until we set up our real page table in mem_init
# in lab 2.
# Load the physical address of entry_pgdir into cr3. entry_pgdir
# is defined in entrypgdir.c.
movl $(RELOC(entry_pgdir)), %eax
f0100015: b8 00 00 11 00 mov $0x110000,%eax
movl %eax, %cr3
f010001a: 0f 22 d8 mov %eax,%cr3
# Turn on paging.
movl %cr0, %eax
f010001d: 0f 20 c0 mov %cr0,%eax
orl $(CR0_PE|CR0_PG|CR0_WP), %eax
f0100020: 0d 01 00 01 80 or $0x80010001,%eax
movl %eax, %cr0
f0100025: 0f 22 c0 mov %eax,%cr0
- 在 0x0010025处下断点, 与预想一致 即 0xf0100000的内存与 0x00100000内存内容相同.
- (由于在该指令之前, 并没有打开内存映射. 所以 0xf0100025实际上还是 0x0010025 )
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
=> 0x100025: mov %eax,%cr0
Breakpoint 2, 0x00100025 in ?? ()
(gdb) x/8x 0xf0100025
0xf0100025 <entry+25>: 0x00000000 0x00000000 0x00000000 0x00000000
0xf0100035 <relocated+6>: 0x00000000 0x00000000 0x00000000 0x00000000
(gdb) x/8x 0x00100025
0x100025: 0xb8c0220f 0xf010002f 0x00bde0ff 0xbc000000
0x100035: 0xf0110000 0x00005fe8 0x55feeb00 0x8353e589
(gdb) si
=> 0x100028: mov $0xf010002f,%eax
0x00100028 in ?? ()
(gdb) x/8x 0xf0100025
0xf0100025 <entry+25>: 0xb8c0220f 0xf010002f 0x00bde0ff 0xbc000000
0xf0100035 <relocated+6>: 0xf0110000 0x00005fe8 0x55feeb00 0x8353e589
(gdb) x/8x 0xf0100025
0xf0100025 <entry+25>: 0xb8c0220f 0xf010002f 0x00bde0ff 0xbc000000
0xf0100035 <relocated+6>: 0xf0110000 0x00005fe8 0x55feeb00 0x8353e589
Syntax Highlighting Post
Syntax highlighting is a feature that displays source code, in different colors and fonts according to the category of terms. This feature facilitates writing in a structured language such as a programming language or a markup language as both structures and syntax errors are visually distinct. Highlighting does not affect the meaning of the text itself; it is intended only for human readers.
Pygments Code Blocks
To modify styling and highlight colors edit 1
/assets/less/pygments.less
1
main.less
1
main.css
1
.highlight
#container {
float: left;
margin: 0 -240px 0 0;
width: 100%;
}Line numbering enabled:
1
2
3
4
5
6
7
8
<nav class="pagination" role="navigation">
{% if page.previous %}
<a href="{{ site.url }}{{ page.previous.url }}" class="btn" title="{{ page.previous.title }}">Previous article</a>
{% endif %}
{% if page.next %}
<a href="{{ site.url }}{{ page.next.url }}" class="btn" title="{{ page.next.title }}">Next article</a>
{% endif %}
</nav><!-- /.pagination -->
module Jekyll
class TagIndex < Page
def initialize(site, base, dir, tag)
@site = site
@base = base
@dir = dir
@name = 'index.html'
self.process(@name)
self.read_yaml(File.join(base, '_layouts'), 'tag_index.html')
self.data['tag'] = tag
tag_title_prefix = site.config['tag_title_prefix'] || 'Tagged: '
tag_title_suffix = site.config['tag_title_suffix'] || '–'
self.data['title'] = "#{tag_title_prefix}#{tag}"
self.data['description'] = "An archive of posts tagged #{tag}."
end
end
endStandard Code Block
1
2
3
4
5
6
7
8
<nav class="pagination" role="navigation">
{% if page.previous %}
<a href="{{ site.url }}{{ page.previous.url }}" class="btn" title="{{ page.previous.title }}">Previous article</a>
{% endif %}
{% if page.next %}
<a href="{{ site.url }}{{ page.next.url }}" class="btn" title="{{ page.next.title }}">Next article</a>
{% endif %}
</nav><!-- /.pagination -->
Fenced Code Blocks
To modify styling and highlight colors edit 1
/assets/less/coderay.less
1
main.less
1
main.css
1
.coderay
1
_config.yml
1
coderay
1
2
3
4
5
#container {
float: left;
margin: 0 -240px 0 0;
width: 100%;
}
1
2
3
4
5
6
7
8
<nav class="pagination" role="navigation">
{% if page.previous %}
<a href="{{ site.url }}{{ page.previous.url }}" class="btn" title="{{ page.previous.title }}">Previous article</a>
{% endif %}
{% if page.next %}
<a href="{{ site.url }}{{ page.next.url }}" class="btn" title="{{ page.next.title }}">Next article</a>
{% endif %}
</nav><!-- /.pagination -->
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
module Jekyll
class TagIndex < Page
def initialize(site, base, dir, tag)
@site = site
@base = base
@dir = dir
@name = 'index.html'
self.process(@name)
self.read_yaml(File.join(base, '_layouts'), 'tag_index.html')
self.data['tag'] = tag
tag_title_prefix = site.config['tag_title_prefix'] || 'Tagged: '
tag_title_suffix = site.config['tag_title_suffix'] || '–'
self.data['title'] = "#{tag_title_prefix}#{tag}"
self.data['description'] = "An archive of posts tagged #{tag}."
end
end
end
great article about troublshooting a server in first five minutes
I didn’t write this, i found the post a few days ago and i think it can be very helpful, so i make a copy here.
First 5 Minutes Troubleshooting A Server
Back when our team was dealing with operations, optimization and scalability at our previous company, we had our fair share of troubleshooting poorly performing applications and infrastructures of various sizes, often large (think CNN or the World Bank). Tight deadlines, “exotic” technical stacks and lack of information usually made for memorable experiences.
The cause of the issues was rarely obvious: here are a few things we usually got started with.
Get some context
Don’t rush on the servers just yet, you need to figure out how much is already known about the server and the specifics of the issues. You don’t want to waste your time (trouble) shooting in the dark.
A few “must have”:
- What exactly are the symptoms of the issue? Unresponsiveness? Errors?
- When did the problem start being noticed?
- Is it reproducible?
- Any pattern (e.g. happens every hour)?
- What were the latest changes on the platform (code, servers, stack)?
- Does it affect a specific user segment (logged in, logged out, geographically located…)?
- Is there any documentation for the architecture (physical and logical)?
- Is there a monitoring platform? Munin, Zabbix, Nagios, New Relic… Anything will do.
- Any (centralized) logs?. Loggly, Airbrake, Graylog…
The last two ones are the most convenient sources of information, but don’t expect too much: they’re also the ones usually painfully absent. Tough luck, make a note to get this corrected and move on.
Who’s there?
1
2
$ w
$ last
Not critical, but you’d rather not be troubleshooting a platform others are playing with. One cook in the kitchen is enough.
What was previously done?
1
$ history
Always a good thing to look at; combined with the knowledge of who was on the box earlier on. Be responsible by all means, being admin shouldn’t allow you to break ones privacy.
A quick mental note for later, you may want to update the environment variable 1
HISTTIMEFORMAT
What is running?
1
2
$ pstree -a
$ ps aux
While 1
ps aux
1
pstree -a
Listening services
1
2
3
$ netstat -ntlp
$ netstat -nulp
$ netstat -nxlp
I tend to prefer running them separately, mainly because I don’t like looking at all the services at the same time. 1
netstat -nalp
1
numeric
Identify the running services and whether they’re expected to be running or not. Look for the various listening ports. You can always match the PID of the process with the output of 1
ps aux
We usual prefer to have more or less specialized boxes, with a low number of services running on each one of them. If you see 3 dozens of listening ports you probably should make a mental note of investigating this further and see what can be cleaned up or reorganized.
CPU and RAM
1
2
3
4
$ free -m
$ uptime
$ top
$ htop
This should answer a few questions:
- Any free RAM? Is it swapping?
- Is there still some CPU left? How many CPU cores are available on the server? Is one of them overloaded?
- What is causing the most load on the box? What is the load average?
Hardware
1
2
3
$ lspci
$ dmidecode
$ ethtool
There are still a lot of bare-metal servers out there, this should help with;
- Identifying the RAID card (with BBU?), the CPU, the available memory slots. This may give you some hints on potential issues and/or performance improvements.
- Is your NIC properly set? Are you running in half-duplex? In 10MBps? Any TX/RX errors?
IO Performances
1
2
3
4
$ iostat -kx 2
$ vmstat 2 10
$ mpstat 2 10
$ dstat --top-io --top-bio
Very useful commands to analyze the overall performances of your backend;
- Checking the disk usage: has the box a filesystem/disk with 100% disk usage?
- Is the swap currently in use (si/so)?
- What is using the CPU: system? User? Stolen (VM)?
1
dstat
Mount points and filesystems
1
2
3
4
5
6
7
$ mount
$ cat /etc/fstab
$ vgs
$ pvs
$ lvs
$ df -h
$ lsof %2BD / /* beware not to kill your box */
- How many filesystems are mounted?
- Is there a dedicated filesystem for some of the services? (MySQL by any chance..?)
- What are the filesystem mount options: noatime? default? Have some filesystem been re-mounted as read-only?
- Do you have any disk space left?
- Is there any big (deleted) files that haven’t been flushed yet?
- Do you have room to extend a partition if disk space is an issue?
Kernel, interrupts and network usage
1
2
3
4
5
$ sysctl -a | grep ...
$ cat /proc/interrupts
$ cat /proc/net/ip_conntrack /* may take some time on busy servers */
$ netstat
$ ss -s
- Are your IRQ properly balanced across the CPU? Or is one of the core overloaded because of network interrupts, raid card, …?
- How much is swappinness set to? 60 is good enough for workstations, but when it come to servers this is generally a bad idea: you do not want your server to swap… ever. Otherwise your swapping process will be locked while data is read/written to the disk.
- Is
1
conntrack_max
- How long do you maintain TCP connections in the various states (
1
TIME_WAIT
1
netstat
1
ss
Have a look at Linux TCP tuning for some more pointer as to how to tune your network stack.
System logs and kernel messages
1
2
3
4
$ dmesg
$ less /var/log/messages
$ less /var/log/secure
$ less /var/log/auth
- Look for any error or warning messages; is it spitting issues about the number of connections in your conntrack being too high?
- Do you see any hardware error, or filesystem error?
- Can you correlate the time from those events with the information provided beforehand?
Cronjobs
1
2
$ ls /etc/cron* %2B cat
$ for user in $(cat /etc/passwd | cut -f1 -d:); do crontab -l -u $user; done
- Is there any cron job that is running too often?
- Is there some user’s cron that is “hidden” to the common eyes?
- Was there a backup of some sort running at the time of the issue?
Application logs
There is a lot to analyze here, but it’s unlikely you’ll have time to be exhaustive at first. Focus on the obvious ones, for example in the case of a LAMP stack:
- Apache & Nginx; chase down access and error logs, look for
1
5xx
1
limit_zone
- MySQL; look for errors in the
1
mysql.log
- PHP-FPM; if you have php-slow logs on, dig in and try to find errors (php, mysql, memcache, …). If not, set it on.
- Varnish; in
1
varnishlog
1
varnishstat
- HA-Proxy; what is your backend status? Are your health-checks successful? Do you hit your max queue size on the frontend or your backends?
Conclusion
After these first 5 minutes (give or take 10 minutes) you should have a better understanding of:
- What is running.
- Whether the issue seems to be related to IO/hardware/networking or configuration (bad code, kernel tuning, …).
- Whether there’s a pattern you recognize: for example a bad use of the DB indexes, or too many apache workers.
You may even have found the actual root cause. If not, you should be in a good place to start digging further, with the knowledge that you’ve covered the obvious.
Port mapping/forwarding with IPTABLES in linux
Introduction
We got a problem with the networking in my dorm and laboratory, if outgoing traffic port is 3389, the connection will be banned. so, we need to figure a way out to connect a Windows server’s 3389 port, which is suck, i mean, Windows sucks.
Solution
I connected to one of the linux server which is in the same lan with Suck Windows Server with port, let’s say 63389. the the linux with redirect the connection from 63389 to Suck Windows Server’s port 3389.
this is what i did in the linux server:
1
2
iptables -t nat -I PREROUTING 1 -p tcp -i eth0 --dport 63389 -j DNAT --to-destination SUCK-WIN-IP:3389
iptables -t nat -A POSTROUTING -p tcp --dport 3389 -j MASQUERADE
Note
the second target MASQUERADE is a really handy thing, it will be acting like SNAT most times, but it can also auto-adjust some connection problems with extra COST