ren@hadoop101:/opt/module$ hadoop version Hadoop 3.3.1 Source code repository https://github.com/apache/hadoop.git -r a3b9c37a397ad4188041dd80621bdeefc46885f2 Compiled by ubuntu on 2021-06-15T10:51Z Compiled with protoc 3.7.1 From source with checksum 88a4ddb2299aca054416d6b7f81ca55 This command was run using /opt/module/hadoop-3.3.1/share/hadoop/common/hadoop-common-3.3.1.jar
ren@hadoop101:/opt/module/hadoop-3.3.1$ cat data/dfs/data/current/BP-2027323945-192.168.10.11-1646133053594/current/finalized/subdir0/subdir0/blk_1073741825 test hehe haha test za zhe
ren@hadoop101:/opt/module/hadoop-3.3.1$ hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-3.3.1.jar wordcount /input /wcountput 2022-03-02 15:44:37,508 INFO client.DefaultNoHARMFailoverProxyProvider: Connecting to ResourceManager at hadoop102/192.168.10.12:8032 2022-03-02 15:44:37,882 INFO mapreduce.JobResourceUploader: Disabling Erasure Coding for path: /tmp/hadoop-yarn/staging/ren/.staging/job_1646190571735_0001 2022-03-02 15:44:38,221 INFO input.FileInputFormat: Total input files to process : 1 2022-03-02 15:44:38,302 INFO mapreduce.JobSubmitter: number of splits:1 2022-03-02 15:44:38,450 INFO mapreduce.JobSubmitter: Submitting tokens for job: job_1646190571735_0001 2022-03-02 15:44:38,450 INFO mapreduce.JobSubmitter: Executing with tokens: [] 2022-03-02 15:44:38,561 INFO conf.Configuration: resource-types.xml not found 2022-03-02 15:44:38,562 INFO resource.ResourceUtils: Unable to find 'resource-types.xml'. 2022-03-02 15:44:38,716 INFO impl.YarnClientImpl: Submitted application application_1646190571735_0001 2022-03-02 15:44:38,741 INFO mapreduce.Job: The url to track the job: http://hadoop102:8088/proxy/application_1646190571735_0001/ 2022-03-02 15:44:38,741 INFO mapreduce.Job: Running job: job_1646190571735_0001 2022-03-02 15:44:43,845 INFO mapreduce.Job: Job job_1646190571735_0001 running in uber mode : false 2022-03-02 15:44:43,851 INFO mapreduce.Job: map 0% reduce 0% 2022-03-02 15:44:46,897 INFO mapreduce.Job: map 100% reduce 0% 2022-03-02 15:44:51,958 INFO mapreduce.Job: map 100% reduce 100% 2022-03-02 15:44:52,996 INFO mapreduce.Job: Job job_1646190571735_0001 completed successfully 2022-03-02 15:44:53,065 INFO mapreduce.Job: Counters: 54 File System Counters FILE: Number of bytes read=58 FILE: Number of bytes written=545001 FILE: Number of read operations=0 FILE: Number of large read operations=0 FILE: Number of write operations=0 HDFS: Number of bytes read=128 HDFS: Number of bytes written=32 HDFS: Number of read operations=8 HDFS: Number of large read operations=0 HDFS: Number of write operations=2 HDFS: Number of bytes read erasure-coded=0 Job Counters Launched map tasks=1 Launched reduce tasks=1 Data-local map tasks=1 Total time spent by all maps in occupied slots (ms)=1323 Total time spent by all reduces in occupied slots (ms)=2244 Total time spent by all map tasks (ms)=1323 Total time spent by all reduce tasks (ms)=2244 Total vcore-milliseconds taken by all map tasks=1323 Total vcore-milliseconds taken by all reduce tasks=2244 Total megabyte-milliseconds taken by all map tasks=1354752 Total megabyte-milliseconds taken by all reduce tasks=2297856 Map-Reduce Framework Map input records=6 Map output records=6 Map output bytes=51 Map output materialized bytes=58 Input split bytes=101 Combine input records=6 Combine output records=5 Reduce input groups=5 Reduce shuffle bytes=58 Reduce input records=5 Reduce output records=5 Spilled Records=10 Shuffled Maps =1 Failed Shuffles=0 Merged Map outputs=1 GC time elapsed (ms)=75 CPU time spent (ms)=560 Physical memory (bytes) snapshot=498315264 Virtual memory (bytes) snapshot=4984467456 Total committed heap usage (bytes)=403177472 Peak Map Physical memory (bytes)=287354880 Peak Map Virtual memory (bytes)=2484170752 Peak Reduce Physical memory (bytes)=210960384 Peak Reduce Virtual memory (bytes)=2500296704 Shuffle Errors BAD_ID=0 CONNECTION=0 IO_ERROR=0 WRONG_LENGTH=0 WRONG_MAP=0 WRONG_REDUCE=0 File Input Format Counters Bytes Read=27 File Output Format Counters Bytes Written=32
java.net.UnknownHostException: hadoop102: hadoop102 at java.net.InetAddress.getLocalHost(InetAddress.java:1475) at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(Job Submitter.java:146) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290) at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:415)
微信
支付宝